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ABSTRACT 

We study the clustering properties of z ~ 5.7 Lya emitters (LAEs) in a cosmological reionization 
simulation with a full Lya radiative transfer calculation. Lya radiative transfer substantially modi- 
fies the intrinsic Lya emission properties, compared to observed ones, depending on the density and 
velocity structure environment around the Lya -emitting galaxy. This environment-dependent Lya 
selection introduces new features in LAE clustering, suppressing (enhancing) the line-of-sight (trans- 
verse) density fluctuations and giving rise to scale-dependent galaxy bias. In real space, the contours 
of the three-dimensional two-point correlation function of LAEs appear to be prominently elongated 
along the line of sight on large scales, an effect that is opposite to and much stronger than the linear 
redshift-space distortion effect. The projected two-point correlation function is greatly enhanced in 
amplitude by a factor of up to a few, compared to the case without the environment-dependent selec- 
tion effect. The new features in LAE clustering can be understood with a simple, physically motivated 
model, where Lya selection depends on matter density, velocity, and their gradients. We discuss the 
implications and consequences of the effects on galaxy clustering from Lya selection in interpreting 
clustering measurements and in constraining cosmology and reionization from LAEs. 
Subject headings: cosmology: observations — galaxies: halos — galaxies: high-redshift — galaxies: 

statistics — intergalactic medium — large-scale structure of universe — radiative 

transfer — scattering 



1. INTRODUCTION 

Lya emitters (hereafter LAEs) are galaxies with strong 
Lya emission lines. Owing to the strong Lya line feature, 
LAEs can be efficiently detected through narrowband 
imaging or with integral-field-units (IFU) spectroscopy, 
which makes them natural targets for searches of high- 
redshift galaxies. Large samples of high-redshift LAEs 
are expected with ongoing and forthcoming LAE surveys. 
In this paper, we investigate the clustering of z ^ 5.7 
LAEs in a cosmological reionization simulation, focusing 
on the effects of Lya radiative transfer on their clustering 
properties. 

Large samples of LAEs would provide exciting 
opportunities to probe the high redshift universe. 
The resonance nature of Lya line makes LAEs 
a sensitive probe of the high-redshift intergalac- 
tic medium (IGM), especially across the reion- 
ization epoch (e.g. , 'Miralda-Escud e fc ReesI 119981: 
Miralda-Escud i 119981: FHaiman & Spaa nd 1199'! iSanto; 
2(304; Haiman & Ccn 2005; Wvithe fc Cen 
Malhotra & Rhoads 2004; Kashi kawa et al.l 
Furlanctto ct al. 2006; Diikstra, Li dz. fc Wvi the 2007; 
Diikstra. Wvithe. fc Haimani i2007l; iMcQuinn et al.l 
2007t iMesinger fc Furla^Iettol [20081 llliev et all 120081 



iDaval et al.ll2008ll2009ll2010[ ). In particular, the cluster- 
ing of LAEs at redshift z > 6 can potentially put tight 
constraints on the ionizatio n status of the IGM (e.g., 
Furlanetto et al. 2006; McO uinn et~an 120071 : llliev et alJ 
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Large samples of LAEs should enable accurate mea- 
surements of their clustering. As with galaxy clustering 
in general, the clustering of LAEs encodes useful infor- 
mation of galaxy formation and evolution. We expect 
to learn about the relation between these young galax- 
ies and dark matter halos, and to obtain insights on the 
early stage of structure formation. 

Galaxy clustering also encodes important cosmological 
information. The fluctuation power spectrum of galaxies 
is related to that of matter, usually differing by a con- 
stant, multiplicative galaxy bias factor on large scales. 
Because nonlinearity is weaker at higher redshift, the 
galaxy power spectrum may be used down to smaller 
spatial scales for constraining cosmology, tightening con- 
straints on parameters like the neutrin o mass and provid- 
ing tests of inflation (e.g.. iTakada et al. 2006). The effi- 
cient detection of LAEs at high redshift makes them at- 
tractive candidates in this endeavor. Large-volume sur- 
veys of LAEs, such a s the Hobby-Eb erly Telescope Dark 
Energy Experiment ()Hill et al.ll2008[ ). might also enable 
the d etection of the baryon acoustic oscillations feature 
(e.g.. lEisenstein et al.l[2005[ ) in the LAE power spectrum. 
Baryon acoustic oscillations can be used to measure the 
expansion history of the universe, contributing to con- 
straints on the evolution of dark energy and the curva- 
ture of the universe. 

At present, measurements of LAEs clustering are very 
limite d by small survey volumes and small samples of 
LAEs. lOuchi et al.l ()2003[ ) present the angular two-point 
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correlation function (hereafter, 2PCF) of 87 z = 4.86 
LAEs in a O.lSdeg^ narrowband survey in the Subaru 
Deep Field. The correlation length is estimated to be 
(3.5 ± 0.3)/i-iMpc (and (6.2 ± 0.5)h-^Mpc if a maxi- 
mum contamination correction is applied), larger than 
that of z ^ 4 Lyman break galaxies. With a set of LAEs 
found in a larger area (^ 0.3 deg^) around the same field, 
IShimasaku et al.l ()2004l ) also report strong clustering for 
41 z = 4.86 LAEs, but they find almost no clustering for 
51 z = 4.79 LAEs. The strong clustering of z = 4.86 
LAEs is difficult to reproduce wit h a simple model tha t 
relates LAEs to dark matter halos ()Hamana et al.lf2004[ ) . 
With 151 z = 4.5 LAEs in a 0.36deg2 field of the 
narrowband Large Area Lyman Alpha (LALA) survey, 
iKovac et al.) (|2007l ) estimate a correlation length of 3.2 ± 
0.4/i~^Mpc (4.6±0.6/i"^Mpc with contamination correc- 
tion). The clustering can be reproduced if these LAEs 
reside in halq s more massive than (1-2) x 10^^ h~^MQ. 
iGawiser et all ()2007[ ) measure the angular 2PCF of 162 
z = 3.1 LAEs discovered in a 0.28deg^ field of the 
MUltiwavelength Survey by Yale-Chile (MUSYC). They 
find a moderate clustering with a correlation length 
of 2.5t^:?/i-iMpc, corresponding to that of halos with 
a minimum mass of -2.8xlOi°/i-^M0 (and a median 
mass of 5.6 x lO^^h-^MQ). The 261 z = 2.1 LAEs 
in the same MU SYC field have a c orrelation length of 
3.2±0.6/i-iMpc ()Guaita et al.ll2010D . corresponding to a 
median halo mass of 1.8 X 10"/i-^Mq. From the cluster- 
ing of z = 3-7 LAEs i n the Ideg^ Suhaiu/ XMM-Newton 
Deep Survey fSXDSV lOuchi et all (12011 infer that the 
average host halo mass is 1O"'^''-1O^"'^M0. Understanding 
the LAE clustering results needs to take into account 
the differences in LAE samples and redshifts, as well as 
sample variance caused by small survey volumes. The 
development of a physically based theoretical model of 
LAEs and their clustering is also needed. 

In models of LAEs, the Lya flux of an LAE is usu- 
ally computed from the emitted ionizing photons in the 
galaxy residing in a dark matter halo, assuming Case B 
recombination ()Osterbrocklll989|) . The theoretical mod- 
eling of LAE clustering depends on how LAEs and dark 
matter halos are connected, and how the observed Lya 
luminosity is determined from the intrinsic Lya lumi- 
nosity. Generally speaking, there are two scenarios con- 
sidered in LAE models, the duty cycle scenario and the 
Lya escape fraction scenario. In the duty cycle scenario, 
LAEs are short-lived and at any given time only a frac- 
tion of all galaxies are active as LAEs. In the Lya escape 
fraction scenario, it is assumed that only a fraction of 
Lya photons can escape from the source, and therefore 
the observed Lya luminosity is a fraction of the intrinsic 
one. Either scenario can make the predicted Lya lumi- 
nosity function (LF) match the observation. For a given 
number density of LAEs, the masses of host halos in 
the duty cycle scenario would be on average lower than 
those in the escape fraction scenario. As a consequence, 
the clustering of LAEs would be different in the two sce- 
narios, with a stronger clustering in the escape fraction 
s cenario. 

iNagamine et al.l (|2010[ ) predict LAE clustering based 
on cosmological smoothed particle hydrodynamic (SPH) 
simulation and consider both scenarios. They find that 



LAE clustering measurements from observations are in 
favor of their duty cycle scenario. iTilvi et al.l ([20091 ) 
present an LAE model in which Lya luminosity or star 
formation rate (SFR) is related to the halo mass accre- 
tion rate, rather than halo mass, and the model naturally 
gives rise to the duty cycle of LAEs. Their model pre- 
dict correlati on lengths o f LAE s in agreement with the 
observations. lOrsi et ahl (|2008D combine a semi-analytic 
model of galaxy formation with a large A'^-body simu- 
lation to predict the clustering of LAEs. They adopt 
the scenario of Lya escape fraction and assume the es- 
cape fraction to be constant (2%) and independent of 
galaxy properties. By accounting for the large sample 
variance, the model is found to reproduce the angular 
cluste ring meas u remen ts from current surveys of LAEs. 
McQ uinn et al.l (j2007| ) develop a model of LAEs using 
reio nization sim ulations with cosmological volume (see 
also llliev et ahl [2008) and discuss the effect of reioniza- 
tion on LAE clustering. Their model computes the es- 
cape fraction based on a simplified treatment of Lya ra- 
diative transfer (hereafter, RT) that consists of multi- 
plying the intrinsic line profile by exp(— Tj^), where is 
the optical depth at frequency v along the line of sight. 
The exp(— Ti,) model is reasonably accurate in the case 
of studying the absorption feature of Lya photons pass- 
ing through some neutral regions along the line of sight, 
especially for absorption caused by the damping wing 
or small optical depth. For studying the observed Lya 
emission, the model is not accurate. Even if one limits 
the exp(— r,y) model to study the transfer outside of a ra- 
dius much larger than halo size by assuming a Lya line 
profile at that radius, it is not clear what radius to use, 
what line profile to assume, and what angular distribu- 
tion of Lya emission at that radius to adopt. The model 
neglects the spatial and frequency diffusion of Lya pho- 
tons, which is important because of the scattering, not 
t he absorption nature of the transfer of Lya photons. 

iZheng et al.l ()2010l ) (hereafter Paper I) present a simple 
physical model of LAEs, where Lya RT is the primary 
physical process transforming intrinsic Lya emission 
properties to observed ones. F or the first time, a full RT 
calcu lation of Lya photons (|Zheng fc Miralda-Escudel 
in gas halos around LAEs is performed in a 
self-consistent fashion w ith the ra diation-hydrodynamic 
reionization simulations ()Trac et al. 2008). While in this 
model the number of Lya photons in the IGM is correctly 
conserved, only a fraction of them can be observed, those 
included in the central part of the extended Lya emission 
with high enough surface brightness. The model predicts 
a broad distribution of apparent (observed) Lya luminos- 
ity at fixed intrinsic Lya luminosity or ultraviolet (UV) 
luminosity, a consequence of a variable intergalactic envi- 
ronment of LAEs and the environment-dependent RT of 
Lya photons. Therefore, the model predicts an effective 
Lya escape fraction that is not constant, but has a broad 
distribution and is correlated with the environment. This 
simple physical model is able to ex plain an array of ob - 
served properties of z ^5.7 LAEs in lOuchi et al.l (|2008[ ). 
including Lya spectra, morphology, and apparent Lya 
LF. The broad distribution of apparent Lya luminosity 
at fixed UV luminosity provides a natural explanation 
for the observed UV LF, especially the turnover toward 
the low-luminosity end. The model also reproduces the 
observed distribution of Lya equivalent width (EW) and 
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explains the deficit of UV briglit, liigh-EW sources. 

In this paper, we investigate the clustering of LAEs 
within the model presented in Paper I. As we will show, 
the environment-dependent Lya RT introduces new and 
significant effects in the clustering of galaxies selected by 
Lya emission, a real physical effect that has not been 
properly taken into account in previous studies. We first 
present the environment dependence of the Lya selec- 
tion and the dependence on halo mass in Section [2l In 
Section [3l we present the results of LAE clustering from 
our model, in terms of the 2PCFs. Following an intuitive 
interpretation of the features seen in LAE clustering, we 
provide a simple physical model to further aid our under- 
standing of LAE clustering. In Section |4l we show the 
environment dependence of the halo occupation distri- 
bution (HOD) of LAEs. In Section 5, we summarize our 
main findings and discuss the implications. In the ap- 
pendices, we provide an extended simple physical model 
of LAE clustering, present the power spectrum of LAEs, 
make comparisons to the LAE clustering in the exp(— T,y) 
model, and present tests on factors that may mask the 
new clustering effects. 

Throughout the paper, we adopt the same cosmolog- 
ical m odel as in the reionization simulation (jTrac et al.l 
[2001 used in our RT calculation. It is a spatially flat 
ACDM cosmological model with Gaussian initial density 
fluctuations, and the cosmological parameters are consis- 
tent with the Wilkinson Micro wave Anisotropy Probe 5 
year data punklev et al.l 120091 ): Vim = 0.28, I^a = 0.72, 
Ob = 0.046, h = 0.70, Us = 0.96, and cts = 0.82. Our 
Lya RT calculation is based on the z — b.7 output of 
the simulation, which has a box size of lOO/i^^Mpc on 
a side. In our calculation, a 768"^ grid is used to rep- 
resent the neutral hydrogen density, temperature, and 
peculiar velocity fields in the simulation box. The Hub- 
ble flow is added to the velocity field. LAEs are assumed 
to reside in dark matter halos with positions and veloci- 
ties from the halo catalog. To reduce source blending in 
the Lya image and spectra, Lya photons are collected 
with a finer spatial resolution, a 6144^ grid for the im- 
age of the whole box, corresponding to 16.3/i^^kpc (co- 
moving) or 0.58"per pixel. The spectral resolution and 
range are 0.lA(25kms~^) and 24Ain rest frame, respec- 
tively. We divide the whole simulation box into three 
layers so that the depth of each layer approximates that 
from the width of th e narrowband filte r used in search- 
ing for z - 5.7 LAEs (jOuchi et al.ll2008D . The calculation 
result for each layer is saved in an IFU-like datacube of 
dimension 6144x6144x240. We refer the readers to Pa- 
per I for more details about the characteristics of the 
simulation and calculation. 

2. ENVIRONMENT DEPENDENCE OF Lyo RADIATIVE 
TRANSFER 

As shown in Paper I, a point Lya -emitting source be- 
comes extended in the end as a result of spatial diffusion 
caused by Lya RT. Only the central, high surface bright- 
ness part of the extended source can be observed as an 
LAE. Therefore, the observed or apparent Lya luminos- 
ity (iapparont) is rcduccd with respect to the intrinsic Lya 
luminosity (-Zjintrinsic)- The suppression of Lya emission, 
characterized by the ratio of observed to intrinsic Lya lu- 
minosity, depends on the environments in the vicinities 
of LAEs. The environments here are interpreted broadly 



as the matter density and peculiar velocity. In Paper I, 
we identify a few environment variables. In general, the 
suppression is weaker in regions of lower density; the sup- 
pression depends on the sign of the density gradient along 
the line of sight, in the sense that sources located in the 
near side (with respect to the observer) of an overdense 
region have a lower suppression; relative to the Hubble 
flow, sources moving away from the observer have a lower 
suppression because of the additional redshift in Lya fre- 
quency; sources with larger line-of-sight gradient of the 
line-of-sight peculiar velocity are more easily observed 
because of an effectively larger Hubble expansion rate. 

In Figure [TJ we show the environmental dependence 
of Lya RT outcome in terms of the observed-to-intrinsic 
Lya luminosity as a function of four physical variables 
— overdensity (5, its gradient along the line of sight, 
line-of-sight halo velocity, and peculiar velocity gradi- 
ent along the line of sight, separately for four subsets of 
halos divided according to their halo mass. The over- 
density field is smoothed with a three-dimensional (here- 
after, 3D) top-hat filter of radius 2/i~^Mpc (comoving). 
It is seen that the overall dependence becomes weaker 
for sources in halos of higher mass, probably indicating 
that Lya scatterings encountered in virialized regions or 
their immediate surroundings, which are less affected by 
environments, becomes more important for higher mass 
halos. 

Gravitational evolution of the cosmic structure leads 
to correlations among the above environment variables, 
which complicates the isolation of the role of each vari- 
able on the Lya RT. To have a better understanding 
of the interplay of different environment variables, we 
study in detail the joint dependence of Lya RT on two 
variables, the matter density and the line-of-sight veloc- 
ity gradient, which appear to be the major factors in the 
effects shown in the clustering of LAEs (as discussed in 
the following section). 

The gray scale in Figure [2] shows the joint distribu- 
tion of local matter overdensity 5 and dv^/dz for halos 
of mass ~ lO^°/i~^M0. Clearly, the two quantities are 
anti-correlated: dvz/dz is, on average, larger (i.e., less 
negative) in less dense region, albeit with a large disper- 
sion, which means that there is an effective higher Hub- 
ble constant in less dense regions than in higher density 
regions. This is easily understood using linear theory, 
where the linear velocity field is related to overdensity 
by i5 oc —V • V from the continuity equation. Therefore, 
on average or in the spherical case, a higher density cor- 
responds to a lower value of velocity gradient. Complex 
non-spherical geometry of density perturbations around 
sources causes the large dispersion seen in the correla- 
tion. 

At each position in this dvz/dz-6 plane, we compute 
the median value of the ratio of the observed to intrinsic 
Lya luminosity. The contours show the distribution of 
the median ratio, with thicker contours for higher values. 
In the model with full Lya RT calculation (left panel), 
at a fixed velocity gradient, Lya luminosity appears to 
be less suppressed in regions of higher density. This 
may appear to be counter-intuitive and in contradiction 
with Figure [ija). Naively, we expect more Lya scatter- 
ings and larger spatial diffusion in denser regions, which 
would lower the ratio of observed-to-intrinsic Lya lumi- 
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Fig. 1. — Dependence of Ly« flux suppression of LAEs on density and peculiar velocity, as a function of halo mass. The suppression is 
characterized by the ratio of the apparent (observed) and intrinsic Lya luminosity Lapparcnt/iintrinsic- (a) Dependence on the smoothed 
overdensity field at the source position. The overdensity field is smoothed with a 3D top-hat filter of radius 2/i~^Mpc (comoving). (b) 
Dependence on the density gradient along the Z-direction. The derivative is with respect to comoving coordinate, (c) Dependence on the 
host halo velocity, (d) Dependence on the linear peculiar velocity gradient along the Z-direction. The linear peculiar velocity is obtained 
from the smoothed overdensity field based on the continuity equation (see the text for details). The velocity gradient is put in units of the 
Hubble parameter. Different colors are for LAE host halos of different masses, as labeled in panel (d). The median of the ratio is plotted 
as a solid curve. The two dotted curves delineate the upper and the lower quartiles, and for clarity wc only plot those for the lowest mass 
range. Note that the line-of-sight direction (from the observer to sources) is along the —Z-direction, which matters for interpreting the 
results in panels (6) and (c). See Section [2] for further discussion. 



nosity. This is opposite to what is seen in Figure[2l which 
imphes that the density effect is masked by a stronger ef- 
fect. 

To understand this, we need to note that the linear 
velocity field is related to overdensity, S oc —V ■ v = 
— {dvx/dx + dvy/dy + dvz/dz). Therefore, at fixed 
dvz/dz a higher density corresponds to lower values of 
velocity gradients in the transverse directions. Since ve- 
locity gradient is dominant in determining observed-to- 
intrinsic Lya luminosity ratio (see Section 7 in Paper I), 
lower values of velocity gradients in the transverse di- 
rections indicate a lower probability for Lya photons to 
escape along the transverse directions. This is expected 



from the Lya RT — we can think of this as that the scat- 
terings of Lya photons enable them to probe the optical 
depth in all directions and they prefer to travel the path 
with the least resistance. 

Figure [3] gives a schematic illustration of the depen- 
dence of Lya RT on the surroundings. Lya photons are 
emitted from a central source (marked as a star in each 
panel). The gray scale of the ring around the source in- 
dicates Lya optical depth in each direction, darker for 
higher optical depth. Scatterings of Lya photons tend 
to make them escape in directions with lowest optical 
depth, leading to higher fiux in these directions. The 
lengths of arrows illustrate the magnitude of fiux in each 
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Fig. 2. — Joint dependence of Lya flux suppression of LAEs on environments in the fuU Lyo RT model (left) and the exp(— r^) model 
(right). In each panel, the gray scale shows the distribution of halos (~ 10^''h~^MQ) in the plane of density and line-of-sight velocity 
gradient, darker for higher probability density. Contours indicate the median Lya flux suppression, defined as the ratio of observed to 
intrinsic hya luminosity, for LAEs residing in these halos. Thicker contours correspond to higher ratios, and adjacent contours differ by 0.1 
dex in contour levels. Note the difference in the trend of flux ratios at fixed velocity gradient in the two models. See the text for details. 
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Fig. 3. — Schematic illustration of RT effect on the observed fiux 
of hya emission. In each panel, Ly« photons are emitted from the 
central source and scattered by neutral gas surrounding it. The 
gas is shown as the shaded region, and the gray scale indicates 
the hya optical depth along each radial direction with darker for 
higher optical depth. The fiux of escaped Lya photons along each 
direction is illustrated by the arrows with length proportional to 
flux. Although the optical depths along the line-of-sight direction 
(from the observer to the source) are the same in all four pan- 
els, we expect to have different observed flux because of different 
distribution of optical depth in other directions. 



direction. Although the line-of-sight optical depth is the 
same and the total flux over all directions is the same in 
the four cases shown in Figure [Sj the observed fluxes are 
not. For example, in panel (d), the optical depths are 
high in directions other than the line-of-sight direction, 
and Lya photons are all likely to be reflected toward the 
observer. While in panel (c), the optical depths are low 
in directions other than the line-of-sight direction, and 
the observer can only see a very low flux of Lya photons. 
The observed flux is therefore not purely determined by 
the line-of-sight optical depth, but the line-of-sight opti- 
cal depth relative to those in all other directions. This 
is a fundamental difference between the exp(— r^) model 
and the full Lya RT model, with the former only using 
the line-of-sight information. In the exp(— T,y) model, 
at fixed line-of-sight velocity gradient, the optical depth 
should increase with density, which is opposite to the 
full RT model and is seen in the right panel of Figure [5J 
The relative amplitude of the optical depth is the zeroth- 
order, primary effect on the directional propagation of 
Lya photons. Of course, the overall amplitude of the op- 
tical depth (e.g., in the spherically symmetric case) de- 
termines the spatial extent of the Lya photon diffusion 
and the overall degree of suppression of the observed Lya 
luminosity. 

While at fixed dv^/dz the observed-to- intrinsic Lya 
luminosity ratio increases with density, it decreases with 
density if averaged over environments, as seen in Fig- 
ure[Ua) . The apparent contradiction is resolved by notic- 
ing that the latter is driven by the strong dependence of 
the Lya luminosity ratio on dvz/dz (Figure [TJd) and the 
anti-correlation between density and dvz/dz (gray scale 
in Figure [2]). 

To have a visual impression of the environmental de- 
pendence of Lya flux suppression in the simulation, 
we show a slice (5/i~^Mpc thick) of the spatial dis- 
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Fig. 4. — Spatial distribution of LAEs. The slice is 5/i ^Mpc thick. The gray scale map shows the matter density distribution (smoothed 
at 2/i~^Mpc scales; darker for higher density) to delineate the large-scale filamentary structures. Points represent halos above 5x 10^/i~^ A/q . 
Left: distribution of LAEs as a function of the ratio of the apparent (observed) and intrinsic Lyci luminosity Lapparcnt/^intrinsic. Rsd 
(blue) points are for sources with Lyo luminosity weakly (strongly) suppressed, with the apparent to intrinsic luminosity ratio higher than 
14% (lower than 0.6%), which approximately corresponds to the top (bottom) 10% of the luminosity ratio distribution. Right: distribution 
of LAEs (red points) above a threshold in apparent (observed) luminosity. The luminosity threshold corresponds to a LAE number density 
of 3.8 X lO^^/i^Mpc""^. Note that the line-of-sight direction (from the observer to sources) is along the — Z-direction. That is, the distant 
observer observes the sources from the top of the panels, which matters for interpreting the relation between Lya flux suppression and the 
local environment (see the text). 



tribution of LAEs in Figure HI In our Lya RT cal- 
culation, the observational direction is along the —Z- 
direction, i.e., the observer is assumed to observe from 
the top of the plot. Points denote all the halos more 
massive than 9 x lO^/i~^M0. In the left panel, red 
points represent sources with weak Lya flux suppression 
(high ratio of Lapparcnt/iintrinsic) and blue points are for 
sources with strong Lya flux suppression (low ratio of 
^apparent /-^intrinsic). One Can scc the environmental de- 
pendence in the sense that sources in underdense regions 
(larger line-of-sight velocity gradient) have a higher prob- 
ability of being less suppressed in Lya flux, e.g., sources 
around the void at (X, Z) = (35, 20). One can also see 
the line-of-sight density gradient effect. Sources on the 
near side of an overdense region are more likely to be less 
suppressed in Lya flux, e.g., sources above and below the 
overdense region near (X, Z) = (30, 40). 

In the right panel of Figure U red points represent 
LAEs above a threshold in the observed Lya luminos- 
ity. The luminosity threshold corresponds to a sample of 
LAEs with number density of 3.8 x IQ-'^h^yipc^'^ . The 
red points therefore show the spatial distributions of ob- 
served LAEs. They appear to be strongly clustered with 
large voids between them. In the next section, we study 
the clustering of LAEs in more detail. 

3. THE CLUSTERING OF LAEs 

In this section, we start with the clustering of LAEs in 
one sample and the associated control samples to show a 
few new features in galaxy clustering introduced by Lya 
RT. Then we present a simple physical model to aid our 
understanding of these new features. With this model in 



hand, we study LAE clustering in detail, including the 
projected/ angular clustering and the dependence on ob- 
served Lya luminosity. We present the clustering results 
from our simulation in terms of the 2PCFs. The fluc- 
tuation power spectra for a couple of LAE samples are 
shown in Appendix [B] A comparison between the pre- 
dictions of LAE clustering in the full Lya RT and the 
exp(— Ti.) models is presented in Appendix [Cl 

3.1. Real-space and Redshift- space 2PCFs of LAEs 

To mimic observations, we construct LAE samples 
based on the apparent (observed) Lya luminosity, with 
each sample consisting of all LAEs with observed Lya 
luminosity above a threshold. For such luminosity- 
threshold samples, luminosity threshold is uniquely 
mapped onto sample number density. Hereafter, we de- 
note different samples by the number densities of LAEs. 

To better illustrate the environment-induced effects, 
we also construct control samples that are supposed to 
be free of environment effects induced by Lya RT. We 
generate two types of control samples for each LAE sam- 
ple. The first type is a mass-threshold sample of halos 
that has the same number density as the LAE sample. 
This sample corresponds to a model of LAEs that relates 
observed LAEs to halos through a one-to-one map of ob- 
served Lya luminosity onto halo mass (a.k.a. abundance 
matching) . 

The second type of control sample is called the shuffled 
LAE sample (S-LAEs) . The sample is constructed as fol- 
lows. We first sort halos by mass and divide them into 
narrow mass bins with typical bin width of 0.02-0.04 dex 
in log M . The number of halos in each bin is a few thou- 
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Fig. 5. — Spatial distributions of LAEs and control sources in real space, with one spatial direction being along the line of sight. For 
LAEs (left panel), the observer is supposed to be on the top of the panel, observing toward the — Z-dircction. The slice in each panel is 
from the same part of the box projected onto the X-Z plane, with a thickness of 20/i~^Mpc. The two control samples are the halo sample 
(middle panel) and the shuffled LAE sample (S-LAEs; right panel), with the same number density (10~^h'^Mpc~'^) as the LAE sample. 
The LAE/S-LAE and halo samples are defined by thresholds in observed Lya luminosity and halo mass, respectively. The shuffled LAE 
sample is expected to eliminate the effect of environmental dependence of Lya RT. Compared with the two control samples, the LAE 
sample shows a prominent pattern of elongated distribution along the line of sight, a result of environment-dependent Lya RT. See the 
text for details. 
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Fig. 6. — Same as Figure (5] but for distributions in the transverse plane perpendicular to the line of sight. In the transverse direction, 
the LAE sample appears to be more strongly clustered than the two control samples, which is a result of environment-dependent Ly« RT. 
See the text for details. 



sands (hundreds) around I0^°h-^MQ (10"/i-1A/q). For 
halos in each narrow mass bin, the Lya emission prop- 
erties (apparent/intrinsic Lya luminosity, Lya spectra, 
and even the UV luminosity) as a whole are randomly 
shuffled among them. Such a shuffling algorithm does 
not lead to any change in the Lya LF, UV LF, and the 
relation between observed to intrinsic Lya properties, 
which keeps the interpretations to the observed proper- 
ties of LAEs presented in Paper I unchanged. However, 
the shuffling is supposed to get rid of any correlation be- 
tween observed Lya properties and environments, with 
the latter encoded in halo positions. From this shuf- 
fled catalog, we form the S-LAE sample above the same 
threshold in observed Lya luminosity as in the LAE sam- 
ple. By construction, the S-LAE sample has the same 



number density as the LAE sample. For the S-LAE sam- 
ple, the dependence of Lya RT results on environment is 
completely eliminated while the statistical properties of 
Lya luminosity is kept, making it a more suitable control 
sample than the halo sample. Similar shuffling method is 
applied in studying environment effe c t on halo clustering 
(assembly bias ; e.g.. lYoo et alll2006l: l&oton et al.ll2007l: 
However, we emphasize that what we 



intend to study here is not the effect of environments on 
halo clustering, whi ch is small for galaxy-host halos at 
high redshifts fe.g.. iWechsler et al.lll200a iGao fc White! 
|2007[) . but the effect of the environment-dependent Lya 
RT on LAE clustering. 

We perform nine realizations of shuffling with different 
random seeds. For each LAE sample, nine S-LAE sam- 
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Fig. 7. — Three-dimensional (3D) two-point correlation functions of LAEs and halos as a function of the line-of-sight separation (rp) 
and the transverse separation (tt). From top to bottom are those of LAEs, halos, and shuffled LAEs (S-LAEs). All samples are threshold 
samples with the same number density (10~^/i^Mpc~^), with the thresholds being the apparent (observed) Lya luminosity for LAEs/S- 
LAEs samples and halo mass for the halo sample, respectively. From left to right are the two-point correlation functions in real space, 
redshift space, and apparent redshift space. In real space, the position of a source is just its comoving position. In redshift space, the 
position is determined by the source's comoving position and peculiar velocity. The apparent redshift is defined from the observed peak 
in Lya spectra, which has the additional shift caused by Lya RT (see the text). It is used to determine a source's position in apparent 
redshift space. The solid contour in each panel denotes a contour level of unity, and the adjacent contours differ by 0.2dex in contour levels. 
Note the prominent elongation pattern along the line of sight in the clustering of LAEs in all three spaces considered here. See the text. 



pies are constructed from these realizations. The sta- 
tistical results for S-LAEs (e.g., correlation functions) 
presented in this paper are always the average over the 
nine realizations. 

In Figure [5l the real-space spatial distributions in the 
X-Z plane of the LAE sample and the two correspond- 
ing control samples are compared. All the three sam- 
ples have number density of 10~^/i^Mpc~'^. By the term 
"real space" , we mean to use the comoving coordinates 
of sources, with no peculiar velocity displacements. Each 
slice has a thickness of 20/i"^Mpc. The line-of-sight di- 
rection is along —Z. Unlike sources in the control sam- 
ples, the spatial distribution of LAEs shows a distinct 
pattern stretching along the line of sight. The elonga- 



tion resembles the fingers-of-God (FoG) effect seen in 
redshift-space galaxy clustering. However, it clearly dif- 
fers from the FoG effect. First, the elongation effect for 
the LAE sample shows up in real space. Second, while 
the FoG effect is on scales of virialized halos, the elonga- 
tion effect for the LAE sample exists up to much larger 
scales. 

For the two control samples, there is no elongation pat- 
tern. The clustering of the S-LAE sample appears to be 
weaker than the halo sample. Although the two samples 
have the same number density, there are S-LAEs resid- 
ing in halos of mass lower than the mass threshold of 
the halo control sample (see Section 0]). Since low-mass 
halos are less clustered than high-mass halos, the S-LAE 
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sample displays a weaker clustering pattern. 

In Figure El the real-space spatial distributions of 
sources in the X-Y plane (perpendicular to the line of 
sight) are compared for the three samples. For the two 
control samples, the trend is the same as in Figure [5l as 
expected. The LAE samples show much stronger cluster- 
ing than the control S-LAE sample. For this particular 
set of samples, it is also true that LAEs are more strongly 
clustered than halos in the halo control sample. 

The different effects of Lya selection on the line-of- 
sight and transverse distributions of LAEs means that 
the spatial distribution (hence the clustering) of LAEs is 
no longer isotropic even in real space. We measure the 3D 
2PCFs of the LA E and control saruples u sing the Landy- 
Szalay estimator (|Landv fc Szalavlll993f) . The real-space 
3D 2PCF ({rp.Tr) for the above LAE sample (top-left 
panel of Figure [7|) clearly shows the anisotropy in LAE 
clustering, where Vp and tt are the transverse and line- 
of-sight separations of galaxy pairs, respectively. The 
contours are elongated along the line-of-sight separation. 
The scales range from sub-Mpc to > 10/i~^Mpc, where 
the 2PCF can be accurately measured. The 2PCFs of 
sources in the halo and S-LAE control samples (middle- 
left and bottom-left panels) appear to be isotropic, mani- 
fested by contours being circular. Note that the deviation 
from circular contours at large separation for the control 
samples is caused by statistical fluctuations (sample vari- 
ance) with the finite simulation box. 

In redshift space, there is a well-known effect (Kaiser 
effect; .Kaiser., 1987.) that introduces anisotropy in galaxy 
clustering on linear scales, which makes the 2PCF con- 
tours squashed along the line of sight. The linear Kaiser 
effect is clearly seen for the control samples in redshift 
space (the two lower panels in the middle column of Fig- 
ure [71). However, for the LAE sample, the prominent 
elongation pattern along the line of sight is preserved in 
redshift space, with a slight reduction in the degree of 
the elongation. 

In the above discussion, the redshift-space source po- 
sitions are calculated as the sum of comoving coordi- 
nates and the apparent position change introduced by 
the line-of-sight peculiar velocity. In practice, if redshifts 
of LAEs are determined from Lya line peak in spectra, 
there would be an additional shift in the apparent posi- 
tion. As shown in Paper I, the observed Lya line peak 
generally shifts redward as a consequence of RT and the 
shift is correlated with the environment. We name the 
redshift measured from Lya line peak as apparent red- 
shift and the corresponding redshift space as apparent 
redshift space. The 2PCFs in the apparent redshift space 
(right column in Figure ^ look similar to those in the 
(true) redshift space (middle column in Figure [71). Since 
at a fixed (true) redshift the apparent redshift has a dis- 
tribution, the 2PCF contours in apparent redshift space 
are slightly smoothed. In all the following discussions, 
when we refer to redshift-space measurements, we use ap- 
parent redshifts for LAE and S-LAE samples and (true) 
redshifts for halo samples, unless mentioned explicitly. 

We see that our RT model of LAEs predicts that 
the clustering of observed LAEs differ significantly from 
those of halos and S-LAEs, which are constructed to be 
immune to the environment effect on Lya RT. Before 
presenting more results in LAE clustering, it is useful 
to gain some understanding of the basic features in the 



clustering of LAEs, for which we give an intuitive picture 
and introduce a simple physical model. 

3.2. Understanding the Clustering of LAEs 
3.2.1. An Intuitive Picture 

The new features appearing in LAE clustering are 
related to the selection imposed by Lya RT, which 
is closely related to the environment around the LAE 
sources. Since the main feature in the 2PCF is a line-of- 
sight distortion, bearing some analogy to redshift-space 
distortion, we start from an intuitive understanding of 
LAE clustering by making comparisons to redshift-space 
distortion. 

The effect of reds hift-space distortion in the linear 
regmie ()Kaiseii[T98^ is illustrated in Figure [SJa). The 
solid black curve shows the real-space line-of-sight den- 
sity distribution of a sample of galaxies, with one mode 
of the plane-wave fiuctuation considered. The arrows 
indicate peculiar velocities of galaxies with the length 
representing the amplitude. In general, galaxies fall into 
overdense regions and stream out of underdense regions. 
When mapped onto redshift space, the line-of-sight den- 
sity distribution of galaxies changes because of the addi- 
tional redshifts caused by peculiar velocities. As a result, 
in redshift space, overdense regions become more over- 
dense and underdense regions become more underdense, 
as shown by the red curve. Redshift-space distortion only 
takes effect along the line of sight, which leads to an en- 
hancement of the line-of-sight fluctuations. In terms of 
the 3D 2PCF, the effect shows up as contours squashed 
along the line of sight. 

For LAEs, the dependence of Lya RT on environments 
leads to a selection function that determines whether a 
source can be detected as LAEs given the flux limit in 
observation. Figure [8l[b) illustrates such a selection ef- 
fect. The black curve is the real-space line-of-sight den- 
sity distribution of an underlying population of LAEs 
(e.g., all the halos that could be detected as LAEs given 
the fiux limit in observation, if there were no Lya RT 
effect). As shown in Section [21 the line-of-sight veloc- 
ity gradient has the strongest effect in determining the 
observed-to-intrinsic Lya luminosity ratio, larger gradi- 
ent for higher ratio. For the plane-wave fluctuation in 
Figure [Hl^b), the velocity gradient reaches its maximum 
and minimum at the trough of the underdense region 
and the peak of the overdense region, respectively. As 
a result, sources in the underdense region have a higher 
probability to be observed as LAEs than those in the 
overdense region, making the overdense region less over- 
dense and the underdense region less underdense (i.e., 
suppressing the line-of-sight fiuctuation). The density 
distribution of LAEs that can be observed is shown as 
the solid red curve (note that we also add a phase shift to 
reflect the dependence of the selection on density and ve- 
locity gradients). The dotted red line is the mean density 
of LAEs, and the difference with respect to the dotted 
black line is an overall selection effect caused by Lya 
RT. The illustration shows that the Lya selection effect 
is similar to redshift-space distortion, but it originates in 
real space with an opposite sign. 

Figure [SI illustrates how to understand the selection ef- 
fect from the point view of angular distribution of Lya 
emission. In Figure HJa), the grays cale delineates a 
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Fig. 8. — Illustration of effects of redsfiift-space distortion and hya RT selection on the observed density fluctuation alone the line of 
sight, (a): Effect of redshift-space distortion. The solid black/thin curve is the real-space density distribution with the dotted line denoting 
the mean. Arrows represent the linear peculiar velocity with the length proportional to the amplitude. The red/thick curve represents the 
density distribution in redshift space, (b): Effect of hya RT selection. The solid black/thin curve is the real-space density distribution of a 
population of galaxies. The observed LAEs are a fraction of these galaxies, with a selection function imposed by the Lyo RT process. The 
density distribution of the observed LAEs in real-space is represented by the solid red/thick curve. The selection favors sources in regions 
with low density and positive line-of-sight velocity gradient and to a smaller degree, in regions with receding line-of-sight velocity that are 
on the near-side (far-side) of overdense (underdense) regions. The observer is assumed to lie on the left side of the panel. See the text for 
details. 
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Fig. 9. — Illustration of the Lya RT selection effect in the cases with fluctuation modes parallel and perpendicular to the line of sight. 
Panel (a) shows the case of density fluctuation along the line of sight. The gray scale delineates the overdensity across one wavelength of 
a plane wave, darker for higher overdensity. Each point represents a hya emitting source, and the ring around it illustrates the angular 
distribution of escaped hya emission. In the underdense region, hya emission preferentially comes out along the line-of-sight direction (and 
its opposite direction), mainly because of the effect of velocity gradient on the RT (see the text). Sources in the underdense (overdense) 
region have a higher (lower) probability to be observed than those in the overdense region. Therefore, the Lya RT selection causes a 
suppression of the fluctuation along the line of sight. Panel (b) is a 90° rotation of panel (a), corresponding to the case of density 
fluctuation perpendicular to the line of sight. In this case, the Lya RT selection causes an enhancement in the fluctuation. See the text 
for details. 



plane- wave density fluctuation along the line of sight, 
darker for higher density. Each dot is an underlying LAE 
source. The ring around each source shows the angular 
distribution of the Lya emission that can be observed. 
For example, in the underdense region (white part of 
the gray scale map), we have a large velocity gradient 
along the line of sight and zero along the transverse di- 
rection. As a consequence, Lya photons preferentially 
escape from the directions parallel and anti-parallel to 
the line of sight, leading to higher surface brightness in 



these directions. While in the overdense region, the situ- 
ation changes to the opposite. For an LAE survey set by 
a Lya flux limit, sources in the underdense region have 
a higher probability to be observed as LAEs. Again we 
reach the same conclusion as above — the line-of-sight 
density fluctuation is suppressed for LAEs. Note that 
except for sources residing at the peak and trough of 
the (plane-wave) density fluctuations, the angular distri- 
bution of Lya emission generally does not have a par- 
ity symmetry along the line of sight. For example, Lya 
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photons in sources near the boundary between the over- 
dense and underdense regions in Figure |9] have their Lya 
photons preferentially escape toward the direction of un- 
derdense region, as a result of the density gradient and 
receding velocity effect. 

For the selection effect in the case of a transverse fluc- 
tuation, we only need to rotate FigurelHIa) by 90°, which 
leads to Figure |9jb). In the underdense region, Lya pho- 
tons appear to preferentially escape in the direction per- 
pendicular to the line of sight. So sources in the un- 
derdense region now have a lower probability to be de- 
tected as LAEs. This leads to the interesting result that 
the density fluctuation in the transverse direction is en- 
hanced for LAEs. 

Note that large-scale density filaments are a common 
feature of gravitational dynamics. Filaments may be de- 
scribed by Figure m The net result is that filaments ori- 
ented along the line of sight are preferentially observed, 
whereas those perpendicular to the line of sight are sup- 
pressed. This feature is clearly seen in the left panel of 
Figure [S] 

Based on the above discussion, the main selection 
effects in galaxy clustering caused by environment- 
dependent Lyck RT can be summarized as the suppres- 
sion of the fluctuation in the line-of-sight direction and 
the enhancement of the fluctuation in the transverse di- 
rection. In what follows, we present a simple physical 
model to describe these effects. 

3.2.2. A Simple Physical Model 

For the simple physical model, we limit our discussion 
to the linear regime. Given that the effects of Lya se- 
lection share some similarities with the Kaiser effect, we 
first review the formalism o f redshift-space distortion in 
galaxy clustering (see, e.g.. lHamilton|[l998l) . We denote 
quantities in redshift space with a superscript "s" . 

The linear density fluctuation Sg{r) of galaxies in real 
space may be related to that of matter 6,n{r) by Sg{r) = 
bS„i (r) , where b is the large-scale galaxy bias factor that 
in general may be scale dependent. From conservation of 
galaxy pairs, the redshift-space density fluctuation Sg{s) 
satisfies 

[l + 5l{s)]d^s^[l + Sg{v)]d^r, (1) 

with s r + Vzz/{Ha), where Vz is the line-of-sight pe- 
culiar velocity and H is the Hubble constant at the time 
when the scale factor is a. Equation ([1]) reduces to 



illustrated in Figure [SJa). In terms of the power spec- 
trum, we have 



" ' Ha dz 



(2) 



The relation is easier to study in Fourier space by 
noticing that 5m = X^k exp(ik • r) and dvz/dz = 
- Ek fHa{kl/k'^)5mM exp(ik • r), 



5gM + /M^'5m,k = (1 + 5, 



g,k- 



(3) 



In the above expression, ~ kz/k is the cosine of the an- 
gle between the line of sight and the wave vector of the 
Fourier mode of interest, / = dlnG/dlna is the deriva- 
tive of the linear growth factor G, and /? = //& is the 
redshift-distortion parameter. The underlying picture of 
the above equation is that galaxies fall into overdense re- 
gions and stream out of underdense regions, enhancing 
the line-of-sight density fluctuation in redshift space, as 



P,«(k) = (l + /3M')^P,(k). 



(4) 



In general, whenever the detection of an object or 
the measurement of a quantity in redshift space is af- 
fected or altered by the gradient of peculiar velocity, 
there are two different bias factors, one related to den- 
sity and the other related to peculiar velocity. The /3 
fact or in the Kaiser formula m ay n ot have its usual m ean- 
ing. iMcDonald etall (poM ) and iMcDonaldl (pOOl dis- 
cuss this for the case of Lya forest correlation, where the 
optical depth or transmitted flux is affected by peculiar 
velocity gradient. In our case, the selection function of 
the galaxies varies with the velocity gradient. 

For LAEs, the density and line-of-sight peculiar veloc- 
ity and their line-of-sight gradient, as the primary envi- 
ronmental factors, affect Lya RT, which imposes a se- 
lection function for the appearance of LAEs. For the 
simple physical model presented here, we only consider 
the two main factors, line-of-sight peculiar velocity gra- 
dient and density. As shown below, including them is 
able to explain the main effects in LAE clustering. For 
completeness, in Appendix [Al we add the other two fac- 
tors into the model and show that their main effect is to 
introduce scale-dependent bias. 

We consider a simple case that the selection function is 
a linear function of the line-of-sight peculiar velocity gra- 
dient dvz/dz and the matter density 6m- In this model, 
the real-space density of LAE galaxies is related to the 
matter density modifled by the selection function. 



lg{l + dg) = gno(l + bSm) x 



1 + aiSm + 0!2 



1 dvz 
Ha dz 



(5) 

where b is the bias factor for the underlying galaxy pop- 
ulation (with mean number density no) before the Lya 
selection is imposed, fig is the mean number density of 
galaxies that are selected as LAEs, q is the overall frac- 
tion of galaxies that are selected as LAEs, and a^ (i=l 
and 2) are dimensionless coefficients (assumed to be con- 
stant). Based on discussions related to Figure [51 the 
Sm term in the selection function represents a combined 
effect of the dependence of Lya RT on Sm and the trans- 
verse peculiar velocity gradient, and both coefficients ai 
and a2 are expected to be positive. Although it is more 
appropriate to separate the terms of density and the 
transverse peculiar velocity gradient (see Appendix \^ , 
we choose to use a positive ai to denote the combined 
effect here for simplicity. 

Keeping the first order terms in Equation ([5]) and notic- 
ing that hg = qfio, we have 



(6) 



(7) 



1 dv 

5g = {b + a,)5m + a, — ^. 
In Fourier space, the relation becomes 

5gM = [{b + ai) - a2/^^] Sm,k- 

Since dvz/dz is the major variable in shaping the 
observed-to-intrinsic Lya luminosity ratio, the coefficient 
a2 is expected to be greater than ai (also see Figure [5]). 
Equation ([7]) describes that the effect of Lya RT selection 
from the velocity gradient term (a2 term) is to suppress 
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the fluctuation along the hne of sight, as long as 012/ is 
not so large to reverse the phase of the fluctuation. 

Adding the Kaiser effect to the above Lya RT selection 
effect, in redshift space, the relation for the total effect 
reads 



[(& + ai) + (l-a2)/M'] Sr, 



(8) 



With Equation (l8|), the power spectrum of LAEs in 
redshift space is then 

P;(k) = [(1 + ^) + (1 - a2)f3^^']'b'P,n{k). (9) 

This full expression of power spectrum includes both red- 
shift distortion and Lya selection effects. The usual red- 
shift space linear power spectrum is a special case with 
the coefffcients ai set to zero. The dependence of Lya RT 
on the velocity gradient leads to the same form of angular 
dependence as the redshift distortion in the anisotropic 
power spectrum, but with an opposite sign. 

The power spectrum in Equation ^ can be decom- 
posed into monopole, quadrupole, and hexadecapole mo- 
ments, 



(l-a2)/3 
b'^Pmik), (10) 



P2ik) = 
and 



(1 + ^) (l-a2)/3 + ^(l-a2)2/3^ 



^4(fc) = — (l-a2)'/3'fo'Pm(fc). 
35 



(11) 

(12) 



The changes in these multipole moments with respect to 
the redshift-distortion-only case (a^ = 0) can be clearly 
seen. The monopole and quadrupole are affected by both 
Lya selection factors, while the hexadecapole only has 
additional contributions from the dependence on velocity 
gradient. 

The coefffcients in Equation © represent the magni- 
tude of the Lya selection effect. As shown in Figure [71 
the elongation pattern along the line of sight is clearly 
seen in redshift space. Since this is mainly caused by 
the velocity gradient term, it means that a2 > 1. The 
ai term represents the combined effect of density and 
transverse velocity gradient. We can imagine that there 
are cases that the dependence on density itself is strong. 
For example, at the reionization stage when ionizing bub- 
bles do not percolate yet, there are isolated ionized and 
neutral regions. If the clustering of sources in overdense 
regions makes these regions ionize earlier, we expect a 
strong dependence of Lya observability on density (a 
large ai), which would substantially enhance the clus- 
tering of LAEs (Equation ([9])). We will examine this in 
more detail in a subsequent investigation of Lya RT at 
higher redshifts where the neutral fraction of the IGM is 
higher and its fluctuation larger. 

In practice, it is hard to measure the 3D clustering 
of LAEs, since it needs a large spectroscopic sample 
of LAEs. The 2D clustering, in particular the angular 
2PCF, is relatively easy to measure with narrow-band 



surveys. The angular 2PCF is closely related to the pro- 
jected 2PCF Wp, which is the 3D 2PCF ^{rp,!:) inte- 
grated over the line-of-sight separation. 



Wp{rp) 



-j-oo 



^(rp, 7r)d7r. 



(13) 



The projected 2PCF is the 2D Fourier transform of the 
power spectrum with the li ne-of-sight w ave vector set 
to zero (see Equation (4) of IZhend 120041 ). From equa- 
tion ([S]), the projected 2PCF of LAEs corresponds to the 
following 2D power spectrum 



P,(kp)=(l + ^ 



b -Prn(kp), 



(14) 



where kp is the wave vector in the plane perpendicular 
to the line of sight. Since ai > 0, Equation means 
that the transverse fluctuation is enhanced for LAEs, as 
schematically shown in Figure ^ 

This highly simplified model enables us to see how dif- 
ferent environment factors related to Lya selection af- 
fect the power spectrum of LAEs. A realistic model is 
expected to be more complicated. For example, the sim- 
ple model assumes that the selection function is a linear 
combination of environment variables. From Figure [1] 
we see that the selection does not have a simple linear 
dependence on any of the factors. In fact, the observed- 
to-intrinsic Lya luminosity ratio approximately shows 
an exponential dependence on environmental variables. 
Furthermore, the selection depends on halo mass. The 
simple model is nevertheless useful for understanding fea- 
tures in the clustering of LAEs, and it is able to describe 
the main effects of Lya selection on LAE clustering. The 
form of the power spectrum in Equation ([9]) and in Ap- 
pendix [A] may be a starting point for a fitting formula. 
In what follows we study LAE clustering in more detail 
and use the simple model to aid our understanding. 

3.3. 3D 2PCFs of LAEs 

In this section, we present the 3D 2PCFs for threshold 
LAE samples. For each LAE sample, we compare the re- 
sults to those of the two threshold control samples (halos 
and S-LAEs) of the same number density. 

Figure [10] shows the real-space 3D 2PCFs as a func- 
tion of sample number density. The 2PCFs of all LAE 
samples clearly show the elongation pattern along the 
line-of-sight direction, even though that of the n — 
2 X lO^'^/i'^Mpc"'^ sample is somewhat noisy. The con- 
tours of the 2PCFs for the control samples appear to 
be circular except for the statistical fluctuation at large 
scales. 

In redshift space, the elongation pattern in the 3D 
2PCFs of LAEs is still preserved (Figure [TT|) , implying 
that the Lya selection effect is much stronger than the 
redshift-space distortion effect. For control samples, the 
contours of 2PCFs are still close to circular. Control 
samples are only affected by the redshift-space distortion 
effect. Halos or S-LAEs at z — 5.7 are highly biased, so 
the redshift distortion parameter (5 w Vl^ /h ~ 1/5 is 
small (see Equation ([J])), which explains the weak red- 
shift distortion seen in the 2PCFs of control samples. 

For all the samples considered here, the 3D 2PCFs of 
LAEs are characterized by elongation contours on scales 
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Fig. 10. — Real-space 3D two-point correlation functions C(?'p, tt) for threshold samples. The top column shows the case for LAE samples. 
For each LAE sample, the middle and bottom columns show the cases for two types of control samples — halos and shuffled LAE samples 
(S-LAEs), both having the same number density as the corresponding LAE sample. The shuffled LAE sample is expected to eliminate 
the effect of environmental dependence of Lya RT (see the text for details). For LAE and S-LAE samples, thresholds in observed Lyo 
luminosity are used to select sources, while for halos, the thresholds arc in halo mass. The number density (in units of h'^Mpc"^) of each 
set of samples is labeled at the top of each column. 



ranging from ^ IMpc to a few tens of Mpc, a com- 
pletely new phenomenon in galaxy clustering predicted 
by our model. If the anisotropic 3D clustering can be 
measured from LAE surveys, it will be a direct test of 
our model and the Lya RT selection effect. Since the vol- 
ume of the simulation we use is 10^/i~'^Mpc'^, Figure [TT] 
implies that a few hundred LAEs with spectroscopic red- 
shifts are needed to have decent measurements of the 
3D 2P CFs. Current LAE surveys, such as the LALA 
survey (IKovac et al.| l2007t IWang et all [20091) and SXDS 
(jOuchi et al. 2008), are approaching this requirement. 

Figures [T^] and [T3] show the spherically averaged 3D 
2PCFs of LAEs and control samples as a function of 
number density. The error bars are obtained with the 
jackknife method, whereby we divide the simulation box 
into eight octants and measure the 2PCFs by excluding 
one octant at a time. The control samples are halos and 
S-LAEs in the two figures, respectively. These plots allow 
a comparison of the overall amplitudes of 2PCFs. In 
each panel, points are the spherically averaged real-space 
3D 2PCFs and curves are averaged from redshift-space 
3D 2PCFs. The difference between the real-space and 
redshift-space average is small, which again demonstrates 
that the redshift-distortion effect is weak for sources at 
z = 5.7. 

The spherically averaged 2PCFs of halos appear to be 
slightly steeper than those of LAEs (Figure[T2|). On large 
scales (~ 1 - lO/i-^Mpc), LAEs are more clustered than 
halos for high number density samples, a consequence 



of the enhancement of LAE clustering by Lya selection. 
At number density 10~^h^Mpc~^ , the amplitude of the 
2PCF of halos exceeds that of LAEs. The trend of the 
relative amplitude between LAE and halo samples re- 
flects the competition of the Lya selection effect and halo 
bias. At fixed number density, LAEs can populate into 
halos of mass lower than the mass threshold that de- 
fines the halo control sample. One would naively expect 
that LAEs should be less clustered than halos. However, 
the selection effect from Lya RT boosts the clustering of 
LAEs (Equation ^ and Equation (ITU)) ). So we see that 
LAEs are more clustered than halos for samples with 
high number density. To explain the relative amplitude 
at the low number density end, we need to note that halo 
bias factor is a steeply rising function toward low number 
density (i.e., toward more massive and rare halos). At 
very low number density, the boost from Lya selection 
in LAE clustering can no longer catch up with the steep 
increase in halo bias factor, which results in the reversal 
of the relative 2PCF amplitude. 

The S-LAEs, on the other hand, seem to always have a 
lower amplitude in the spherically averaged 2PCF than 
the LAEs of the same number density (Figure [T5|) . S- 
LAE samples are fairer control samples than halo sam- 
ples, since they share the same statistical properties of 
Lya emission with the LAE samples except for the en- 
vironmental effect on Lya RT. The LAE and S-LAE 
samples of the same number density have the same un- 
derlying population. Setting the coefficients ai and a2 
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Fig. 11. — Same as in Figure [TOl but for redshift-space 3D two-point correlation functions. 



to zero in the simple physical model (e.g., Equation ([9|) 
and Equation ([TUl) ) corresponds to the cases of S-LAEs. 
From the S-LAE and matter 2PCFs in Figure [H the 
bias factor b is of the order of 10, so the redshift distor- 
tion parameter /3 is at the level of 0.1. The monopole of 
the power spectrum (Equation (I10|) ) is likely to be domi- 
nated by the first term, which is from the density depen- 
dence. The LAE sample therefore always has enhanced 
clustering with respect to the corresponding S-LAE sam- 
ple. As the bias factor b in Equation ((TU)) increases 
with decreasing sample number density, we expect the 
enhancement effect to become weaker at lower number 
density, consistent with what is seen in Figure [131 For 
n = IQ-^h^M-pc"^ , the 2PCFs of the LAE and S-LAE 
samples are quite close to each other. 

3.4. Projected/ Angular 2PCFs of LAEs 

The angular 2PCFs of LAEs can be measured with 
LAEs identified in narrow-band surveys. On scales 
smaller than the length scale of the redshift range of the 
survey, which is set by the width of the narrow-band 
filter, the angular 2PCF is closely related to the pro- 
jected 2PCF, or the projected power spectrum (Equa- 
tion (HH)). Here we present the projected 2PCFs of LAE 
samples and the corresponding control samples in Fig- 
ure [T4]( with the error bars in the LAE sample estimated 
from the jackknife method). 

The Lya RT effect boosts the transverse density fluc- 
tuation, leading to an enhancement in the projected 
2PCFs. In the simple physical model, the enhancement 
comes from the density dependence of the selection func- 
tion (Equation p4)) ). which is an effective description 
of the complicated dependences of RT on density and 



line-of-sight and transverse velocity gradients (see Ap- 
pendix The trends in the relative amplitudes be- 
tween the projected 2PCFs of LAEs and halos and be- 
tween the projected 2PCFs of LAEs and S-LAEs are sim- 
ilar to those seen in the spherically averaged 2PCFs, as 
shown in Figure 1141 

To quantify the amplitude of the projected 2PCFs of 
LAE and control samples, we compute an average bias 
factor of LAEs, S-LAEs, and halos for each sample num- 
ber density. We measure the projected 2PCF of matter 
from the matter density field in the simulation (dashed 
curves in Figure [HI) . The average bias factor of LAEs/S- 
LAEs/halos is taken to be the square root of the ratio of 
the projected 2PCFs of LAEs/S-LAEs/halos and mat- 
ter, averaged over the range of l-10/i~^Mpc. It turns 
out that the ratio is not constant at the above scales, 
being smaller toward larger scales. The error bars we as- 
sign to the average bias factors largely reflect this scale 
dependence. 

The left panel of Figure [TSj shows the bias factors of 
LAEs, S-LAEs, and halos, as a function of sample num- 
ber density. Plotted in the right panel are the bias factors 
of LAEs/S-LAEs relative to halos. At the same num- 
ber density, S-LAEs can occupy halos with mass lower 
than the threshold mass of the halo sample, therefore S- 
LAE samples always show a lower bias factor than halos. 
The relative bias factor of S-LAEs and halos is roughly 
a constant (~0.8) for n = 10"2-10"^ft.^Mpc"^. It de- 
creases toward lower number density and reaches ^0.6 
at n = lO-^/i^Mpc"^ 

It is interesting that the bias factor for LAE samples 
does not show strong dependence on sample number den- 
sity. The value is around 10 for sample number den- 
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Fig. 12. — Spherically averaged two-point correlation functions of LAEs as a function of number density. In each panel, filled circles 
are the real space two-point correlation function of LAEs, and open triangles are that for the halo control sample with the same number 
density. LAE and halo samples are defined by thresholds in observed Lya luminosity and halo mass, respectively. The number density 
(in units of /I'^Mpc"^) is marked in the panel. The solid and dotted curves are from redshift space for LAEs and halos, respectively. The 
dashed curve is the two-point correlation function of matter. 




0.1 1 10 0.1 1 10 0.1 1 10 

r (h-^Mpc) r (h-'Mpc) r (h-^Mpc) 

Fig. 13. — Same as Figure [121 with control samples changed to shuffled LAE samples (S-LAEs; open circles and dot-dashed curves). In 
each panel, LAE and S-LAE sample are defined by the same thresholds in observed Lya luminosity. Shuffled LAE samples are expected 
to eliminate the effect of environmental dependence of Lyo RT (see the text for details). 



16 



103 

Oh 

S 102 



1 — I I I I iiij 1 — I I I 1 1 ii| 

n = 5xl0-3 -»- LAEs 

A- Halos 
A-a -e- S-LAEs 




^ ^ a, AN 
J I I H' ^a>.i 



T — I I I I iiij 1 — I I I 1 1 ii| 

n = 5xl0-4 




I I r ~. . I "11 



= III 


1 1 iij 


1 1 1 1 1 Illj 


1 d 






n = 2xl0- 


3 I 










A - A -A- 


~ A 








-g""A 










A -^N^ 








^ A 








■S AW 








^ "«a\- 








" ^ ^ ^''^ 












III 


1 1 III 


i1~ 


?'l 1 




T — I I 1 1 1 ii| 1 — I I 1 1 1 ii| 

n=lxl0-3 




J I I 





1 1 1 1 1 1 II 




1 d 


: o-o 'a. 


.n=lxlO- 


4 = 


\ J 


A, 






'\\ 


m_'A 








■i- G^J\ ^ 






















1 


i" 







0.1 



10 



Tp (h"'Mpc) 



Tp (h"^Mpc) 



Tp (h"^Mpc) 



Fig. 14. — Projected two-point correlation functions Wp of LAEs as a function of number density. In each panel, besides the projected 
two-point correlation function of LAEs (filled circle), we also plot those for the halo control sample (open triangles) and the shuffled LAE 
sample (S-LAEs; open circles) of the same number density (marked in the panel in units of /I'^Mpc"^). LAE/S-LAE and halo samples 
are defined by thresholds in observed Ly« luminosity and halo mass, respectively. The dashed curve is the projected two-point correlation 
function of matter, scaled by a factor of 10. 
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FlG. 15. — Bias factors of LAEs, shuffled LAEs (S-LAEs), and halos as a function of sample number density. Left: absolute bias 
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Fig. 16. — Prediction of the an gular two-po i nt cor relation func- 
tion for the z ~5.7 LAE sample in lOuchi erall 1)20081 ). The thresh- 
old sample has a number density of 1.2 X 10~^/i^Mpc~^. The 
predicted angular two-point correlation function of this sample of 
LAEs is shown as filled circles. The two dotted curves represent the 
1(T scatter expected in a survey with SXDS-hke volume. Triangles 
and open circles are the two-point correlation functions of halos 
and shuffled LAEs with the same number density as the LAE sam- 
ple. The dashed curve is the matter correlation function, scaled by 
a factor of 10. 
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Fig. 17. — Sample variance of the angular two-point correlation 
function for z ~5.7 LAE sample in SXDS-like volume. The sam- 
ple has a nu mber density of ~ 2 X 1 0~^fe'^Mpc~^, comparable to 
the sample inliVlurayama e t al.l II2007I V Filled squares, crosses, and 
open squares are the angular two-point correlation functions mea- 
sured in three fields from the simulation, showing a large sample 
variance. Filled circles with error bars are the mean of the three 
measurements. The dotted curve is the best-fit power law to the 
measured LAE two-point correlation function in lMuravama et al.l 
1(20071). 

sity in the range of 10~^-10~^/i'^Mpc~'^ and increases 
slightly toward the low number density end. However, 
the bias of LAEs relative to halos continuously decreases 
toward low number density (filled circles in the right 
panel), from ~1.75 at n = lO^^/i^Mpc"^ to ~0.7 at 



n = lO-^/i^Mpc"^. 

The weak dependence of LAE clustering on sample 
number density is a result of the selection effect caused 
by the environment dependent Lya RT. In the simple 
physical model, the amplitude of the projected 2PCF is 
proportional to (b+ai)'^ (Equation (fT4|) ). As the number 
density decreases, the bias factor b increases. The weak 
dependence of the amplitude on number density would 
require that ai drop with decreasing number density, 
which seems to be consistent with the halo mass depen- 
dence seen in Figure[TJ The prediction can be tested with 
LAE clustering measurements as a function of observed 
Lya luminosity. In Appendix |D1 we show how the weak 
dependence changes if there is a large dispersion in the 
intrinsic Lya luminosity and halo mass relation. 

When interpreting observational results of galaxy clus- 
tering, luminosity-threshold samples of galaxies are usu- 
ally modeled as mass-threshold halo samples for a sim- 
ple inference of the relation between galaxies and ha- 
los. If the same exercise were applied to Lya-luminosity- 
threshold samples of LAEs, our model would have in- 
teresting implications. For high number density sam- 
ples {n > 5 X lO-^/i^Mpc"^), LAEs are more strongly 
clustered than halos in the corresponding mass-threshold 
halo control sample. From the clustering amplitude, one 
would relate the LAE sample to a halo sample with 
higher mass threshold. However, the number density of 
these halos are less than the LAE sample, which would 
imply that there have to be multiple LAEs per halo. 
For example, the bias factor of n = 2 x lO^^ft.^Mpc"'^ 
LAEs is similar to that of n = 4 x IQ-^h^Mpc^^ halos 
(left panel of Figure [T5|) . From the clustering amplitude, 
we would infer that these LAEs reside in halos above 
2 X 10"/i-1Mq (n = Ax IQ-^/i^Mpc"^), and compari- 
son of the halo number density and LAE number density 
would make us infer that there are on average five LAEs 
per halo. If one LAE per halo were assumed, from the 
number density alone, we would infer that these LAEs 
reside in halos above 7 x IO^'^H'^Mq. For low number 
density samples (n < 5 x 10~^/i^Mpc~'^), LAEs are less 
clustered than halo samples, and one would infer a duty 
cycle of LAEs by combining the clustering result and 
the number density. We see that the simple modeling of 
LAEs as mass-threshold halo samples can lead to incor- 
rect inference of the relation between LAEs and halos. 
Therefore, a proper interpretation of LAE clustering re- 
quires a careful account of the large Lya selection effect. 

At present, the large st LAE sample a t z = 5.7 
comes from the SXDS (jOuchi et al.ll200l . If we di- 
vided the simulation box along our chosen line of sight 
into three layers of equal size, the depth of each layer, 
33.33/i~^Mpc, is close to that from the width of the 
narrow-band filter used to search for z ■~5.7 LAEs in 
the SXDS, and the area is almost identical to that of the 
survey (Ideg^). Therefore, the simulation box gives us 
three SXDS-like volumes at z ~5.7. 

The photometric z ~5.7 LAE sample in lOuchi et alj 
(|2008() corresponds to a luminosity-threshold sample of 
number density 1.2 x 10~'^/i'^Mpc~^. We construct a 
threshold sample of model LAEs that has the same 
number density and u se the Landy-Szalay estimator 
(|Landv fc Szalavl Il993[ ) to measure the angular 2PCFs 
in the three SXDS-like volumes, respectively. We also 



18 



perform similar measurements for the hafo and S-LAE 
control samples. In Figure [T6l points connected by solid 
curves are the 2PCFs of LAEs, halos, and S-LAEs, re- 
spectively, averaged over the three SXDS-like volumes. 
Similar to the projected 2PCFs, LAEs and S-LAEs have 
the highest and lowest angular clustering amplitudes, re- 
spectively. The two dotted curves show the variance of 
the angular 2PCFs of LAEs from individual measure- 
ments in the three volumes. Even with the large vari- 
ance on large scales, the SXDS clustering measurement 
is likely to reveal that LAEs are more strongly clustered 
t han halos of the same n umber densityH 

iMuravama et al.l ()2007D present an angular 2PCF mea- 
surement of z = 5.7 LAEs in the Cosmic Evolution Sur- 
vey (COSMOS). The sample number density is about 
2 X 10~^ft.^Mpc^'^ and the effective area for the sample 
is 1.86deg^. They find signals at several scales, but with 
low significance. The clustering results for this sample 
in our model are shown in Figure 1171 Apparently there 
is a large variance among sampl es in the three volumes. 
The power-law fit from Murava mTet al.l (|2007l ) (dotted 
curve) falls in b e tween the three measurements. The 
IMuravama et al.l (|2007D sample is only 1.86 times larger 
in effective area than our sample, so we still expect a 
large sample variance. Large sample variance for low 
density samples i n a sm all-area survey are reported by 
IShimasaku et al.l (|2004[ ) at z ~ 5. They measure angular 
2PCFs of LAEs at z = 4.79 and z = 4.86 in a Subaru 
Deep Field of ~0.3deg^ and find clustering signals in one 
sample and the lack of clustering in the other. Based 
on the mean and uncertainty from our simulation data 
for the 2 x 10~^h^Mpc^^ sample, we estimate that a 
survey about 10 times larger is required for a solid de- 
tection (> 3it) of the clustering signal, which means a 
~10 deg^ survey. For the (relatively small) scales con- 
sidered here {rp < 20/i~-^Mpc or < 0.2deg), the survey 
can be composed of either several separated fields with 
a few square degrees each or one single contiguous field. 
With the same total area, the former case would have a 
slightly smaller number of LAE pairs on the above scales 
because of the edge effect, so a single field is slightly 
preferred. The upcoming Subaru Hyper Suprime-Cam 
(HSC) survey will meet such a requirement. 

Based on the above investigation, we conclude that: 
in the SXDS-like field, large sample variance in angular 
2PCFs is expected for LAE samples with low number 
density (e.g., lower than a few times 10 ~'^/t^Mpc~^): for 
the LAE sample in lOuchi et al.l (|2008f ). it is likely that 
the enhancement in the transverse clustering of LAEs 
caused by the Lya selection effect can be detected. If 
future observation did not detect this effect and the null 
detection could be firmly established, it would have pro- 
found implications in our understanding of LAEs. It 
would mean that much stronger effects mask the Lya 
RT selection effect, which could be extremely large scat- 
ter in the distribution of intrinsic Lya properties caused 
by stochastic star formation and/or dust (see further dis- 

^ lOuchi et aLl (2010) present the angular 2PCF of the z ~ 5.7 
LAE sample. On scales above 100", the measurement agrees with 
our prediction. However, below 100", the measured 2PCF is es- 
sentially flattened, which is not seen in their samples at lower and 
higher redshifts. It may be caused by sample variance or may have 
interesting implications, which merits further investigations. 



cussions in Section [5] and Appendix ID)) . 

4. HOD OF LAEs AND THE ENVIRONMENT 
DEPENDENCE 

The HOD framework fe.g.. IBerlind fc Weinbef^[200l 
has become a powerful tool to interpret galaxy cluster- 
ing. It describes the probability distribution of number 
of galaxies of a given type in a halo as a function of 
halo mass, together with the spatial and velocity dis- 
tributions of galaxies inside halos. The HOD framework 
recasts galaxy clustering in terms of the relation between 
galaxies and dark matter halos. The link from galaxies 
to dark matter halos enables informative tests of galaxy 
formation model and tight constraints on cosmological 
parameters from galaxy clustering. 

The basic assumption in the current version of the 
HOD framework is that the statistical properties of 
galaxies only depend on halo mass and is independent 
of the large-scale environments. For normal galaxies 
(i.e., galaxies selected from continuum or optically-thin 
lines) , the assumption of envi ronment- independent HOD 
is supported by theory (e.g., IBerlind fc W einberg" '2002t 
but s ee Zhu et al. 2006), observation (e.g., B lanton ct aLl 
120061 ). and clustering analysis (e.g.. iTinker et al.ll2008[ )7 
Together with the properties of halo population of a 
given cosmology, the halo-mass-dependent HOD can lead 
to a full description of galaxy clustering. However, 
LAEs that are selected from Lya emission suffer from 
environment-dependent Lya RT effect, so the assump- 
tion of environment-independent HOD is expected to 
break down for LAEs. 

Figure [TH] shows the environment dependence of the 
HOD of LAEs. Since the line-of-sight velocity gradient 
is the dominant variable in determining the observed-to- 
intrinsic Lya luminosity ratio, we use it to define the 
environment for the HOD investigation. In each panel, 
the dashed curve (step function) represents the case for 
the halo control sample. The mass threshold can be read 
off from the plot. The solid curve is the mean occupation 
function of LAEs, averaged over all environments. It is 
also the mean occupation function of S-LAEs. 

In our model, LAEs are related to halos, not sub-halos, 
and there is one Lya emitting source per halo, which 
means that we do not have satellite LAEs. This is a 
good approximation, since there are few group-size halos 
at z = 5.7. The mass of the most massive halo in the 
simulation box is ~ 4 x 10^^/i~^Mq, with a virial radius 
of ^ 0.4ft,~^Mpc. Ignoring the (low) satellite fraction 
of LAEs only affects the clustering on small scales (e.g., 
tenths of Mpc) . The mean occupation functions of LAEs 
in Figure [18] are only for central LAEs and reach unity 
at the high-mass end. 

As mentioned in Paper I, in our LAE model, the intrin- 
sic Lya luminosity is proportional to halo mass (Equa- 
tions (1) and (2) in Paper I). If LAEs were selected based 
on intrinsic Lya luminosity, they would be equivalent 
to the halo sample. Observationally, LAEs are selected 
based on their apparent (observed) Lya luminosity. We 
show in Paper I that the environment-dependent Lya RT 
leads to a broad distribution of apparent Lya luminosity 
at a fixed intrinsic Lya luminosity. Observation detects 
LAEs with observed Lya luminosity above a threshold. 
This means that for sources at a fixed intrinsic Lya lu- 
minosity (or halo mass), only a fraction of them can be 
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Fig. 18. — Environment dependence of the halo occupation distribution of LAEs. LAE samples are defined by thresholds in observed Lyo 
luminosity. The left and right panels are for LAE samples with number density of 10"'^ and 10~''fe''Mpc~^, respectively. The line-of-sight 
velocity gradient dvz/dz is used here as the environment indicator and halos are grouped according to the value of dv^/dz. In each panel, 
filled (open) circles show the mean occupation function of LAEs in halos that are in the upper (lower) quartile of the dv^^jdz distribution. 
The solid curve is the mean occupation function of LAEs over all environments. The dashed curve is the mean occupation function of a 
threshold halo sample with the same number density. 



detected as LAEs. The mean occupation function is sim- 
ply this fraction. From low to high halo mass, the mean 
occupation function appears as a ramp from zero to unity 
(soUd curves in Figure [18]). With the occupation num- 
ber below unity, to maintain the same number density 
as the control halo sample, a fraction of LAEs resides in 
halos of mass lower than the threshold mass of the halo 
sample. The cutoff profile of the LAE mean occupa- 
tion function is similar to that of a luminos ity threshold 
sample of continuum-selected gala xies (e.g.. lZheng et alj 
l2005t IZheng. Coil, fc Zehavill2007l) . although the profile 
for LAEs are much more shallow. The shallow cutoffs 
for LAEs and for continuum-selected galaxies can be 
attributed to the same origin: for continuum-selected 
galaxies, the shallow cutoff reflects the scatter in galaxy 
luminosity at a fixed halo mass, while for LAEs the ap- 
parent Lya luminosity has a distribution at fixed halo 
mass. 

The environment dependence of the HOD of LAEs is 
demonstrated by the filled and open circles in Figure [T51 
The filled (open) circles are the mean occupation func- 
tion of LAEs in halos in the upper (lower) quartile of the 
dvz/ dz distribution. Since the observed-to-intrinsic Lya 
luminosity ratio is positively correlated with the velocity 
gradient dvz/dz, a higher fraction of LAEs in halos with 
higher dvz/dz can be detected, which leads to a higher 
mean occupation function of LAEs (filled circles) in these 
halos. The difference between the mean occupation num- 
bers in the upper and lower quartiles of halos are about 
a factor of 2-3 for the two samples in Figure [181 

If large-scale overdensity 5 is added as another envi- 
ronment variable, halos can be grouped according to the 
joint distribution of dvz/dz and 5 (Figure [2]). We con- 
sider two extreme environments. We divide halos in the 
upper quartile of dvz/ dz according to the value of 5 and 
keep halos in the upper quartile of 5 as the first group. 
We then divide halos in the lower quartile of dvz/dz ac- 



cording to the value of 5 and keep halos in the lower 
quartile of 5 as the second group. The two groups of ha- 
los correspond to the upper-right and lower-left corners 
of the shaded region in Figure [2J As the mean observed- 
to-intrinsic Lya luminosity ratios for halos in these two 
groups differ a lot, so does the mean occupation func- 
tions of LAEs in the two group of halos. We find that 
the mean occupation numbers can differ by more than 
one order of magnitude. 

The strong environment dependence of the HOD of 
LAEs adds complexity to the modeling of their cluster- 
ing. In fact, from the difference in the clustering of the 
LAE sample and the halo control sample, we know that 
the simple model of one LAE per halo with a mass thresh- 
old does not work. The difference in the LAE and S- 
LAE clustering means that the environment-independent 
HOD model is not a good model, either. To fully describe 
the features in the LAE clustering, the environment de- 
pendence has to be incorporated into the HOD model. 

5. CONCLUSION AND DISCUSSION 

We investigate the clustering of LAEs, galaxies that are 
selected by their Lya emission, within a physical model 
that fully accounts for the Lya RT. Our model of LAEs 
combines radiation-hydrodynamic cosmological reioniza- 
tion simulations with Monte Carlo RT for Lya photons. 
It is a simple model, which assumes that RT is the single 
factor in transforming intrinsic Lya emission properties 
to observed ones. As previously shown in Paper I, the 
simple model is able to explain an array of observational 
properties of z = 5.7 LAEs. The model has strong pre- 
dictive power, and we predict the clustering properties 
of LAE clustering in this paper. 

Lya RT depends on the circum-galactic and inter- 
galactic environments of Lya emitting galaxies. Lya 
photons emitted from a source at a halo center see a com- 
plex density and velocity structure and an anisotropic 
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optical depth distribution. Lya photons tend to find the 
easiest way out, and the observed Lya emission is usu- 
ahy anisotropic. We emphasize that the observed Lya 
flux at a given line of sight is not purely determined by 
the line-of-sight optical depth, but the line-of-sight optical 
depth relative to those in all other directions (see Sec- 
tion\^. Because of the resonance nature of the Lya line 
and the large scattering cross-section, the observed Lya 
emission is highly sensitive to the local environments, 
including the matter density, line-of-sight velocity, their 
line-of-sight gradients, and the velocity gradients in the 
transverse directions (not all of these environmental vari- 
ables are independent). It is this strong coupling between 
observed Lya emission and environments that gives rise 
to new effects in LAE clustering that need to be taken 
into account in interpreting the clustering of observed 
LAE samples. 

The overall effects in clustering caused by the Lya 
selection are anisotropic clustering and scale-dependent 
galaxy bias. For density fluctuations along the line of 
sight, Lya selection leads to a higher probability in de- 
tecting LAE sources in underdense regions than in over- 
dense regions, so it suppresses the line-of-sight density 
fluctuations. For density fluctuations perpendicular to 
the line of sight, the anisotropic distribution of Lya emis- 
sion makes sources in overdense regions preferentially se- 
lected, hence the transverse density fluctuations are en- 
hanced. Roughly speaking, filamentary or planar struc- 
tures tend to be preferentially selected when they are 
parallel to the line of sight. 

The suppression of line-of-sight fluctuations and the 
enhancement of transverse fluctuations create anisotropy 
in LAE clustering. The iso-contour curves in the 3D 
2PCFs, which reflect LAE pair counts as a function of 
line-of-sight and transverse separations, show a distinct 
elongation pattern along the line of sight. The elonga- 
tion appears on all scales where we have reliable mea- 
surements of the 2PCFs, ranging from sub-Mpc to over 
10 Mpc. The anisotropic pattern is opposite to the lin- 
ear redshift-spacc distortion effect (Kaiser effect), which 
makes contours squashed along the line of sight. We em- 
phasize that the anisotropy caused by Lya selection orig- 
inates in real space. At z — 5.7, the cases we consider in 
this paper, the Lya-selection-induced distortion in LAE 
clustering is much stronger than the linear redshift-space 
distortion. Therefore, even in redshift space the elonga- 
tion pattern along the line of sight in the 3D 2PCFs is 
well preserved. We note that the elongation also differs 
from the FoG effect seen in galaxy clustering, which is 
caused by random motions of galaxies in virialized struc- 
tures. While the FoG effect only shows up in redshift 
space and on small scales (e.g., < IMpc), the Lya se- 
lection effect can appear in both real and redshift spaces 
and on much larger scales. The Lya RT induced features 
are not "Fingers of God" but "Arms of God" . 

The anisotropic clustering induced by Lya selection 
is a completely new phenomenon in galaxy clustering. 
Other than the usual redshift-space distortion (FoG and 
Kaiser effect), there are other form s of anisotrop i c clus - 
tering discussed in the literature. IPadilla et al.l (|2005[ ) 
study the cross-correlation between voids and galaxies. 
In the redshift-space two-point cross-correlation function 
of voids and galaxies, an elongation pattern along the line 
of sight is found on large scales (up to a few times the 



radius of voids in the sample). Unlike the Lya selection 
effect, which is a real-space effect, the phenomenon in 
the void-galaxy cross-correlation is a redshift-space ef- 
fect. It simply reflects that galaxies tend to stream out 
of void regions. The closest analogy to the Lya selec- 
tion effect we study is the effect of orientation -dependent 
galaxy selection investigated bv lHiratal (|2009f ) , in the case 
that galaxies are aligned by large scale tidal fields. Both 
effects can be attributed to an environment-dependent 
surface brightness selection and act in both real and 
redshift spaces. However, th e Lya RT selection effect 
is much stronger. While the IHiratal effect may change 
the clustering at a level of a few percent, the Lya selec- 
tion can change the clustering amplitude by a factor of 
a few, owing to the high sensitivity of Lya RT to local 
environments of galaxies. In addition, unlike the surface 
brightness distribution of stars in galaxies, which largely 
has a parity symmetry (i.e., similar surface brightness if 
viewed at opposite directions), the anisotropic distribu- 
tion of Lya surface brightness does not necessarily have 
any symmetry. 

The 3D anisotropic clustering of LAEs, if measured, 
will be a strong test to the RT model of LAEs. To 
achieve this goal, a large spectroscopic sample of LAEs 
will be needed. With current narrow-band surveys of 
LAEs, however, the clustering measurements are lim- 
ited to the angular 2PCFs, which resemble the projected 
2PCFs. Unlike the redshift-space distortion effect, which 
is largely eliminated in the projected 2PCFs, we find 
that the Lya selection effect is imprinted in the pro- 
jected 2PCFs. Projection keeps the transverse fluctu- 
ations, which are always enhanced by the Lya selection. 
The projected 2PCF of LAEs has a higher amplitude 
than that of the control LAE sample with environment 
effect removed. Our model makes the following distinc- 
tive prediction (see Figure [T5)) : the amplitude of the LAE 
2PCF has a very weak dependence on the observed Lya 
luminosity. The prediction breaks down for faint LAEs 
if a large dispersion (1 dex) between intrinsic Lya lu- 
minosity and halo mass is introduced, which may re- 
sult from stochastic star formation, but it remains valid 
for luminous LAEs (see Appendix iPj) . This is testable 
with large narrow-band surveys. The relation between 
LAEs and underlying halos has interesting properties. 
For LAE samples defined by a threshold in observed Lya 
luminosity, faint LAEs appear to be more strongly clus- 
tered than mass threshold samples of dark matter halos 
having the same number density. The trend is reversed 
for very bright LAEs (with number density lower than 
2 X lO^^/i'^Mpc^'^), where halos are more strongly clus- 
tered than LAEs of the same number density. If an LAE 
sample with low luminosity threshold were simply related 
to a mass threshold sample of halos and one used the ob- 
served LAE clustering amplitude to infer the halo num- 
ber density, one would reach the conclusion that there are 
multiple LAEs per halo. This is not clearly seen from cur- 
rently available clustering measurements yet, probably 
because of the small sample sizes (hence large variance) 
and the existence of contamination from low-redshift ob- 
jects. 

The Lya-selection-induced anisotropic clustering can 
be largely understood by accounting for the dependences 
of the selection on density and line-of-sight velocity gra- 
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dient. Besides these dependences, Lya selection is also 
related to line-of-sight peculiar velocity and line-of-sight 
density gradient, which can give rise to scale-dependent 
galaxy bias, changing the shape of the fluctuation power 
spectrum with respect to matter f Appendices El and [B|) . 
We develop a simple physical model that incorporates 
all the identified environment variables in the selection to 
aid the understanding and interpretation of LAE cluster- 
ing ( Appendix E|. Although the model is able to provide 
qualitative explanations to features in the LAE cluster- 
ing, it is an over-simplified model, which assumes linear 
dependence on environment variables. A more sophisti- 
cated model will have to rely on a detailed study of the 
dependence of Lya RT on environments and an accurate 
description of the statistical relation between observed 
Lya emission and environmental variables. 

As a powerful tool, the HOD framework has been suc- 
cessfully applied to interpret galaxy clustering data from 
many galaxy surveys. For LAEs, the usual assumption 
in the HOD framework that galaxy properties are only 
a function of halo mass and are independent of large- 
scale environments breaks down, as a result of the se- 
lection caused by environment dependent Lya RT. As 
shown in Section 4, the HOD of LAEs is stro ngly de- 
pendent on environments. iHamana et al.l ()2004D perform 
(environment-independent) HOD modeling of the angu- 
lar 2PCF of 2; = 4.86 LAEs in one Subaru Deep Field and 
find that the observed strong clustering cannot be repro- 
duced by the model. Given the large sample variance in 
the small field and crude error estimate of the clustering 
measurement, however, it is not clear whether the failure 
of the model in this case is caused by the neglecting of the 
Lya selection effect. Our study shows that to correctly 
model LAE clustering within the HOD framework, one 
has to extend the framework to incorporate the strong 
environment dependence of Lya selection. 

As discussed in Paper I, the uncertainties in our cur- 
rent model lie in the intrinsic Lya luminosity and spec- 
tra, which deserve detailed investigation. By "intrin- 
sic" , we mean the properties of Lya photons after es- 
caping the ISM. The intrinsic Lya luminosity is related 
to the SFR and the initial mass function of stars. If 
the model (luminosity-threshold) LAEs are matched to 
the observed LAEs in terms of the number density, the 
uncertainty in the intrinsic Lya luminosity is largely re- 
moved in studying LAE clustering. On the other hand, 
the intrinsic spectrum of Lya emission, especially the 
intrinsic line width, is an important factor in determin- 
ing the strength of the Lya selection effect. A larger 
intrinsic line width would lead to a weaker dependence 
of Lya RT on environments (Paper I), which in turn 
would make the effects of Lya RT on LAE clustering 
weaker. For example, we would see a less elongated pat- 
tern in the 3D 2PCF of LAEs with a larger intrinsic line 
width. The intrinsic line shape may also be modified by 
galactic winds, wh i ch can also affect s the Lya RT (e.g., 
iKunth et all 119981: lAtek et al.l 120081) and the effect on 
clustering. If isotropic galactic winds shift the initial Lya 
line by a few hundred kms~^, the coupling of observed 
Lya emission to the environment can be weakened. How- 
ever, galactic winds usually display a collimated bipolar 
pattern, and photons escaping in directions other than 
the bipolar direction may not achieve a large shift. High- 
resolution simulation of individual galaxies with galactic 



wind included is useful for further studying the effect of 
wind on the observational properties of LAEs. See the 
tests and discussions in Appendix |D] Consequently, if 
galactic wind can have a strong effect, the strength of 
Lya-selection-induced clustering effects would allow us 
to potentially constrain the intrinsic Lya line width and 
shape of LAEs, which would otherwise be difficult to dis- 
cern. For low-mass halos (~ 10^'^h~^ Mq) in the simu- 
lation, the grid for hydrodynamic calculation marginally 
resolves the virial radius. The Lya RT for sources in 
these halos may be limited by the resolution. However, 
the infall region (inside the turnaround radius) , which is 
about 5.6 times larger and is important in shaping the 
Lya RT, is well resolved. Therefore, the Lya RT for 
sources in these low-mass halos is not expected to suffer 
the resolution effect significantly. 

Lya RT is a physical process that must exist around 
LAEs. The new effects on LAE clustering, including the 
enhancement in the projected 2PCFs, are strong, which 
means that they cannot be easily masked by other effects. 
If observation showed a null detection of the effects, e.g., 
finding no enhancement in the angular 2PCFs, it would 
have important implications in our study of LAEs. A 
substantial scatter in the intrinsic Lya emission proper- 
ties at fixed halo mass remains as a possibility to reduce 
or even mask the Lya RT selection effect. In our model, 
the intrinsic Lya emission properties, especially the Lya 
luminosity, are tightly correlated with halo mass. The 
Lya luminosity is based on the SFR averaged over lOMyr 
time scale and the star formation prescription does not 
lead to signi ficant scatter in th e SFR at fixed halo mass 
and redshift (|Trac fc Cenll2007l ). To have a large scatter 
in the instantaneous SFR, one needs to introduce a broad 
distribution of the star formation efficiency or make the 
star formation stochastic in halos of fixed mass. Dust 
in the ISM could further enlarge the scatter in the Lya 
luminosity. If in the end the intrinsic Lya luminosity dis- 
tribution at fixed halo mass were much broader than that 
from Lya RT effect (Paper I), the Lya RT selection could 
be masked. In Appendix |D1 we present simple tests on 
the effect of the dispersion in the relation between intrin- 
sic Lya luminosity and halo mass. The dispersion can 
change the clustering amplitude of LAEs of fixed number 
density by including LAEs residing in lower mass halos. 
However, the anisotropic clustering pattern persists even 
if a large dispersion (1 dex in luminosity) is introduced. 
A broad or stochastic Lya luminosity distribution would 
give rise to an effective duty cycle such that only a tiny 
fraction of galaxies are in the Lya emitting phase at a 
given time, which means that at a given number den- 
sity LAEs reside in halos of much lower mass, compared 
to the case of a narr ow intrinsic Lya lumin osity distri- 
bution to start with. iNagamine et al.l ()2010( ) argue that 
a duty cycle scenario provides a reasonable explanation 
to existing LAE clustering measurements. To fully ad- 
dress the magnitude of the distribution and stochasticity 
of star formation efficiency, the relevant processes have 
to be incorporated in the reionization simulation, which 
is limited by our understanding of baryon physics. Po- 
tential consequences, when combined with Lya RT se- 
lection, on LAE clustering deserves detail investigations. 
Strong tests to the model and constraints on different 
processes are expected to come from the Lya LF, UV 
LF, and clustering of LAEs. 
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The completely new effects on LAE clustering due to 
Lya RT add another layer of complexity in modeling 
their clustering. This means that the existence of Lya 
selection effect would complicate the inference of cosmo- 
logical parameters from LAE clustering in large LAE 
surveys. To map from galaxy clustering to dark mat- 
ter clustering, which directly encodes the cosmological 
information, the effects that are commonly considered in 
current galaxy clustering analysis include the redshift- 
space distortion, the nonlinear evolution of structure, 
and the scale-dependent bias induced by halo biasing. 
For Lya selected galaxies, the new effects that add to 
the above list are the anisotropic clustering (opposite to 
and stronger than the redshift-space distortion) and the 
scale-dependent bias induced by environment-dependent 
Lya RT. For our model with z = 5.7 LAEs, the Lya- 
selection-induced anisotropy is overwhelming. 

At lower redshifts z = 2 — 3 we expect the Lya RT 
induced selection effect to continue to operate in the re- 
gions surrounding galaxies, as at z = 5.7 shown here. 
However, at present, it is not clear what the strength of 
the Lya selection effect will be and up to what scales 
the clustering of LAEs is affected at these redshifts. Sev- 
eral competing factors prevents us from having a simple 
guess. On one hand, lower matter density and higher 
UV background at z = 2 — 3 (hence lower neutral hy- 
drogen fraction in the scattering regions), compared to 
z = 5.7, permit easier escape of Lya photons through 
the circum-galactic and inter-galactic gas. On the other 
hand, a lower Hubble velocity may provide a counter- 
ing factor. We reserve a more detailed investigation of 
clustering of LAEs at z = 2 — 3 for a future work. 

The strong environmental dependence of the Lya selec- 
tion, on the other hand, provides a sensitive way to probe 
the late stage of cosmological reionization. With simple 
treatments of Lya RT, it has been shown that Lya emis- 
sion from LAEs can be used to probe the mean hydro- 
gen neutral fraction as a function of redshift (e.g., from 



the Lya LF; 'Malhotra & Rhoads"2004- 'Haiman fc Oct 
[20051 Both Furlanctto ct al. (2006) and McOuin n et al. 
(|2007f l show that the clustering of LAEs is enhanced 
by the patchy reionization. Our current work is lim- 
ited to z = 5.7, when the reionization is completed, 
and we plan to perform full RT modeling of LAEs at 



higher redshifts in a subsequent paper. Because of the 
fast evolution of neutral hydrogen density as reioniza- 
tion proceeds, we would see a rapid change in the dis- 
tribution of the observed-to-intrinsic Lya luminosity ra- 
tio, which would lead to a rapid change in the clustering 
amplitude of LAEs rela tive to the underlying population 
(|Furlanetto et al.ll2006l ). The isolated H II bubbles intro- 
duce a characteristic scale, whic h would result in a scale 
dependence in the bias factor (jFurlanetto et al.l [2006) . 
We also expect that isolated H II bubbles and large-scale 
fluctuations of photoionization rate may introduce addi- 
tional effects in the 3D anisotropic clustering of LAEs. 
As a whole, we expect that different features in the clus- 
tering of LAEs, if detected, would provide a wealth of 
information about reionization. 
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APPENDIX 

A. THE EXTENDED VERSION OF THE SIMPLE PHYSICAL MODEL 

For LAEs, we have identified density, peculiar velocity, and their gradients as the main factors in transforming the 
intrinsic Lya emission properties to the observed ones. The dependence of Lya RT on these quantities imposes a 
selection function for the appearance of LAEs, which affects the clustering of LAEs. In the simple model of LAE 
clustering presented in Section 13.21 for simplicity we only include the density and line-of-sight velocity gradient to 
understand the main features in LAE clustering. For completeness, we add the other factors into the model here and 
follow Section [32] in presenting it. 

We still consider the case that the selection function is a linear function of the environment variables. The real-space 
density of LAE galaxies is related to the matter density modified by the selection function. 
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where b is the bias factor for the underlying galaxy population (with mean number density no) before the Lya selection 
is imposed, fig is the mean number density of galaxies that are selected as LAEs, and q is the overall fraction of galaxies 
that are selected as LAEs, a^ (i=l, 2, 3, 4, and 5) are coefficients (assumed to be constant) in the selection function, 
ru is a length scale introduced to make the coefficients dimensionless and is chosen to be the Hubble radius c/ H when 
the scale factor is a. Note that unlike the simple model presented in Section |3^ here we explicitly include the term 
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(with coefficient 0.3) dependent on the transverse velocity gradient as well as that on the density (with coefficient fii), 
since they represent different physical effects (Section [2]). The untilded and tilded coefficients have a simple relation. 
Since density and the divergence of velocity are connected from the continuity equation ^ + V • v/a = 0, we have 
ai = ai — 0:3/ and a2 — a2 — as (see below). In equation (jAip . ai represent the selection effect purely caused 
by density and we expect it has a negative sign. Since higher line-of-sight velocity gradient corresponds to higher 
apparent-to-intrinsic Lya luminosity ratio, the coefficient 02 should be positive. For reasons discussed in Section [51 
the coefficient 03 is expected to be negative. The 0:4 term is related to the line-of-sight velocity, which is not an 
accurate description. The selection may only depend on the motion of the galaxy relative to its surrounding medium 
that scatters the Lya photons, and a smoothing scale needs to be introduced in a full description. Here we simply 
include a term to gain a rough idea on the possible effect. 
Keeping the first order terms in Equation (jAip and noticing that hg = qfiQ, we have 
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By using S„i = X^k ^"^k exp(ik • r), d5m/dz = «^^'^"i,k exp(ik • r), Vz = X^k */^«^z/^^'5™,k exp(ik • r), and 
dvz/dz = — fHa{k^/k^)6m.u exp(jk • r), we can express the above relation in Fourier space as 
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Rcdshift-spacc distortion contributes to the /i^ term, and in redshift space, it reads 
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Although ai is expected to be negative, the combination {ai — 0,3 f) should be positive, according to Figure [2] Since 
the overall clustering effect is opposite to the linear redshift-space distortion, we expect {0,2 — 0,3) to be positive. 
The dependence of the selection on line-of-sight velocity and line-of-sight density gradient leads to phase shifts in the 

fluctuation. 

With Equation (jA4[) , the power spectrum of L AEs in redshift space is then 
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This full expression of power spectrum includes both redshift distortion and Lya selection effects. Compared to 
Equation ([9]) , it has more terms to represent different physical effects in the selection. Setting all the coefficients a^ to 
zero gives the usual redshift space linear power spectrum. The dependence of Lya RT on the velocity gradient leads 
to the same form of angular dependence as the redshift distortion in the anisotropic power spectrum, but with an 
opposite sign. The dependences on line-of-sight velocity and line-of-sight gradient of density (the 0:4 and terms in 
Equation (|A5[) also contribute to the anisotropy. Furthermore, these dependences lead to scale-dependent bias for the 
power spectrum, with the velocity and density gradient terms dominant on large and small scales, respectively. 
The monopole, quadrupole, and hexadecapole moments of the power spectrum in Equation (jA5l) are 
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The changes in these multipole moments with respect to the redshift-distortion-only case {on = 0) can be clearly seen. 
The monopole and quadrupole are affected by all the Lya selection factors, while the hexadecapole only has additional 
contributions from the dependence on velocity gradient. 



B. POWER SPECTRUM OF LAEs 



In Section [31 we present the LAE clustering results in the form of 2PCFs, which are commonly measured in LAE 
surveys. In the simple model we develop, however, the clustering is easily studied in terms of the power spectrum, the 
counterpart of the 2PCF in Fourier space. To be in parallel with the physical model and for a better comparison, we 
present the power spectrum of model LAEs measured for a couple of samples. 
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Fig. 19. — Redshift-space power spectra of LAEs (left) and S-LAEs (right). The LAE and S-LAE samples are Lyo luminosity-threshold 
samples of number density 10~^/i^Mpc~^. The S-LAE sample eliminates the environment effect of Lya selection. For each column, the 
top panel shows the 3D redshift-space power spectra. Adjacent contours differ by O.ldex in contour levels. The color coded dashed lines 
are four directions of wave vector. The power spectra along the four directions are compared in the bottom panel (dashed curves). The 
solid black curve in the bottom panel arc the spherically averaged power spectrum and the dotted curve is the matter power spectrum. 

For a given sample of LAEs, we derive a density field from the spatial distribution of LAEs using cloud in cell 
weighting. The density field in the simulation box (lOOft-^-'^Mpc on a side) is put into a 768'^ grid, the same size as 
the grid used in our RT calculation. We then perform fast Fourier transform (FFT) with the density grid to obtain 
the fluctuation power spectrum P(k) of LAEs. To correct for the shot noise contribution to the power spectrum 
and the alias effect in the shot noise (IJinell2005D . we generate 16 random catalogs in boxes of lOO/i ^Mpc on a side. 
The number of particles in each random catalog is the same as that of LAEs in consideration. The shot noise power 
spectrum of each random catalog is calculated from FFT. The average shot noise spectrum over the 16 realizations is 
subtracted from the above measured power spectrum of LAEs to obtain the final power spectrum. We also compute 
the power spectrum Pm{k) of matter with FFT from the grid of matter density field. 

The top-left panel of Figure [19] shows the (apparent) redshift-space 3D power spectrum P(k) of the luminosity- 
threshold sample of LAEs with number density 10~^/i'^Mpc~'^. The line-of-sight elongation pattern seen in the 2PCF 
now corresponds to the squashed contours in the power spectrum plot. For comparison, the S-LAE sample of the 
same number density only shows a weak redshift-space distortion pattern (top-right panel of Figure [T9| . with contours 
slightly elongated along the line of sight. For a better comparison and to see the shape of P(k) more clearly, we plot 
the power spectra at different angles with respect to the line of sight in the bottom panels of Figure [12] for the LAE 
and S-LAE samples, respectively. The curves are color coded in the same way as the directions of wave vectors shown 
in the top panels. The solid black curve in each of the bottom panels is the spherically averaged power spectrum. 
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Fig. 20. — Scale dependence of the LAE bias factor. In the left panel, the solid (dotted) curves are the galaxy bias factors of LAEs 
(S-LAEs) at different directions, color coded in the same fashion as in Figure [T9l The scale dependence seen in the S-LAE bias is caused by 
nonlinear structure growth, while that in the LAE bias is a combination effect of nonlinear growth and Lya selection. In the right panel, 
the relative bias factors of LAEs and S-LAEs are plotted and the scale dependence seen here is supposed to be induced only by the Lya 
selection effect. The number density of the luminosity-threshold LAE or S-LAE sample is lO^^/i'^Mpc"^. 
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Fig. 21. — Same as Figure [20l but for samples of number density 10 
Figure [20l to avoid plotting noisy data. 
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For S-LAEs, the Lya selection effect is eliminated and only redshift-space distortion effect is left. Since the sample 
is highly biased, the redshift-space distortion parameter (3 = Q^-^/b is small and the redshift-space distortion effect is 
weak. In the bottom-right panel, we only see small differences in the P(k) at different angles, or at different values 
of /i (the cosine of the angle between the line of sight and the wave vector). The shape of P(k) reflects that of halos. 
It shows slight difference from the matter power spectrum Pm{k), which results from nonlinear structure growth and 
halo biasing. 

For LAEs, the anisotropy caused by Lya selection is very strong (dashed curves in the bottom- left panel). The 
amplitude of P(k) continuously increases as the direction of the wave vector gets closer to the lin e of sight, i.e., as 
the value of /i becomes larger. This is in line with the trend from the physical model (Equation (|A5|) ). The shape 
of P(k) shows a clear departure from that of P,n{k), which is predicted by the physical model as the results of the 
dependence of Lya selection on the line-of-sight velocity and line-of-sight density gradient. The departure is lar ger a t 
smaller scales (larger k), which implies that the effect of line-of-sight density gradient (the as term Equation (jA5|) ) 
likely dominates the scale dependence. 

To clearly see the scale dependence, we plot the galaxy bias factor at different values of /i as a function of scale in 
the left panel of Figure[20l We compute the galaxy bias factor as the square root of P(k)/Pm(/c). For S-LAEs (dotted 
curves), the deviation from a constant bias factor is a result of nonlinear structure growth and halo biasing. For LAEs 
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(solid curves), the deviation is a combination of the above effects and the Lya selection effect. If we take the ratio 
of the bias factors of LAEs and S-LAEs at different values of n, we can remove the effect only caused by nonlinear 
structure growth and halo biasing. The right panel of Figure [20l shows such relative bias factors of LAEs and S-LAEs. 
The scale dependence in this panel is introduced by the selection effect from environment-dependent Lya RT. We can 
see the trend that the scale dependence is stronger at higher values of /i, in a qualitative agreement with the simple 
model (Equation (jA5|) '). 

Figure [2l] shows the scale dependence of the bias factors of LAE and S-LAE samples with a lower number density 
(10~^/i^Mpc~'^). Since the samples correspond to more massive halos, it is not surprising that the scale dependence 
caused by nonlinear structure growth and halo biasing becomes stronger, as seen in the bias factors of S-LAEs (dotted 
curves in the left panel). The Lya-selection-induced scale dependence and the trend with /i can still be clearly seen 
(right panel). Compared with the lO^^/i'^Mpc"^ case, the bias factor of LAEs with the lower number density shows a 
weaker dependence on /i. In the simple model, this can be largely explained by the larger value of b in Equation (jASp . 

The overall trends seen in the power spectrum of LAEs seem to be captured by the extended simple model. Therefore, 
the simple model can be used for qualitative understanding of LAE clustering. 

C. CLUSTERING OF LAEs IN THE cxp{~T„) MODEL 

In this Appendix, we compare the predictions of clustering of LAEs in the full Lya RT model and the exp{—T^) 
model. 

The exp(— Tjy) model is adopted in previous work for the Lya RT calculation, which computes the line-of-sight 
optical depth t^, to a source and modifies the intrinsic Lya emission by the exp(— Ti,) factor. The exp(— Ti,) model 
is reasonably accurate in describing Lya absorption feature for Lya photons passing through neutral clouds/islands, 
especially for absorption caused by the damping wing or small optical depth. In the case of studying the observed Lya 
emission, as shown in Paper I, the exp(— Ti,) model can qualitatively explain some trends in the Lya flux suppression, 
but it cannot provide quantitative results because of the lack of frequency diffusion and spatial diffusion in the model. 
Previous work intends to limit the exp(— r,y) model to study the transfer outside of a radius much larger than halo 
size by assuming a Lya line profile at that radius. Even if this is the case, it is not clear what radius to use, what line 
profile to assume, and what angular distribution of Lya emission at that radius to adopt, which all rely on the Lya 
RT inside that radius. Here we take the original meaning of the exp(— Ti/) model and compute the line-of-sight optical 
depth Ti, all the way to the source. 

As discussed in Section [21 the line-of-sight optical depth is not the single factor in determining the Lya RT, since 
scatterings of Lya photons enable them to probe the optical depth in all directions. The exp(— Tj^) model does not 
fully take into account the environment around the source. It fails to even qualitatively reproduce the trend of the 
Lya fiux suppression with density at fixed line-of-sight gradient, as seen in Figure [2] 

Since exp(— Ti,) model still partially captures the environment effect on Lya RT, the new effects in LAE clustering 
predicted by our full Lya RT model are expected to be more or less seen in this model. Here, we compare the clustering 
properties of LAEs in the full RT and exp(— Tjy) models for samples defined by thresholds in observed Lya luminosity. 
With the same initial setups (i.e., intrinsic Lya luminosity and line profile) as in our full RT calculation, the exp{—T^) 
model underpredicts the observed Lya luminosity (Paper I). We also include a case of exp{—T^) model with higher 
intrinsic Lya line width crinit, corresponding to the circular velocity at halo virial radius. The apparent Lya LF from 
this exp(— Ti,) model roughly match what we obtain from the above full RT model. We construct luminosity threshold 
LAE samples from the three models to have the same number density. 

Figure [22] compares the spatial distributions and the 3D 2PCFs for LAEs of number density 5 x lO^'^ft.'^Mpc^'^ in 
the three models. The line-of-sight elongation pattern is also seen in the exp(— Ti,) models (middle-left and bottom-left 
panels). However, it is not as prominent as in the full RT model and it is weak in the high-Cinit model (bottom-left 
panel). Compared to shuffled samples (not shown here), LAEs in the exp(— r,y) models also show enhanced clustering 
in the transverse plane. However, the clustering is not as strong as in the full RT model (see panels in the middle row). 
In terms of the reals-pace 3D 2PCFs, LAEs show a stronger clustering and a more prominent line-of-sight elongation 
feature in the full RT model than in the exp(— r,y) models. 

Figure [23| compares the projected 2PCFs in the three models for three LAE samples with different number densities. 
The projected 2PCF appears to be flatter in the exp(— Ti,) models than in the full RT model. The shape is more 
parallel to the matter 2PCF, which suggests a weaker scale dependence in the bias factor. LAEs from the high-o-jnit 
exp(— Tj^) model shows a much weaker clustering than those from the full RT model. The large scale bias factor of 
LAEs in the exp(— Ti.) models relative to halos has a similar trend as seen in Figure [TSl 

Our comparison shows that it may be possible to adjust the intrinsic Lya line width in the exp(— Ti,) model to 
approximately match the apparent Lya LF, but the clustering of LAEs would be far off. Alternatively, the intrinsic 
Lya line width could be adjusted to roughly match the clustering, but the apparent Lya LF would be far off. The 
exp(— Ty) model cannot fit both the apparent Lya LF and the clustering of LAE. 

As a whole, the Lya selection effects on clustering are preserved to some extent in the exp(— r,y) model. The reason 
may lie in that the Lya optical depth is strongly dependent on the velocity gradient, which plays a major role in 
determining the anisotropic distribution of Lya emission of a LAE source (see the illustration in Figure [9|) and in 
driving the anisotropic clustering of LAEs (Section 13. 2|) . The line-of-sight optical depth computed in the exp(— r,y) 
model encodes the information of the line-of-sight velocity gradient, so it is not surprising that the model is able to 
qualitatively captures the main effects of Lya selection in LAE clustering. 
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Fic;. 22. - Spatial distributions of LAEs and 3D 2PCFs in real space in the full Lya RT model (top row) and two cxp(— T,y) RT models 
(middle and bottom rows). The LAE samples from the three models are defined by thresholds in observed Lya luminosity so that they 
have the same number density, 5 X 10^'^fe'^Mpc^'\ The exp(— Ti/) RT model in the middle row adopts the same intrinsic Lyo line width 
as the full RT model, which cannot produce an apparent Lya LP that matches that from the full RT model. The cxp(~T^) RT model in 
the bottom row adopts a larger intrinsic Lya line width so that it roughly produces an apparent Lya LF matching that from the full RT 
model. Left panels compare the distributions with one spatial direction along the line of sight. The observer is supposed to be on the top 
of the panels. Middle panels compare the distributions in the transverse plane perpendicular to the line of sight. The corresponding slices 
from the three models are from the same part of the box, with a thickness of 20/i~^Mpc. Right panels compare the real-space 3D 2PCFs 
of LAEs from the three models. The solid contour in each panel denotes a contour level of unity, and the adjacent contours differ by 0.2dex 
in contour levels. 



However, the exp(— Tj^) model excludes the environment information in directions other than the line of sight. 
Furthermore, the effect of velocity gradient in this model is computed with the initial line profile, despite that the line 
profile changes because of frequency difftision caused by scatterings around the source. The model therefore cannot 
give a full description of Lya selection effect on environment and cannot explain the observed LAE properties (e.g., 
Lya LF and clustering) quantitatively and self-consistently. 
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Fig. 23. — Projected 2PCFs of LAEs in the full Lyo RT model and two exp{— r^) RT models. In each panel, the LAE samples from 
the three models are defined by thresholds in observed hja luminosity so that they have the same number density (marked on top of the 
panel in units of h^Mpc~^). The dashed curve is the projected 2PCF of matter, scaled by a factor of 10. The exp(— r^) RT model with 
open circles adopts the same intrinsic Lyo line width as the full RT model, which cannot produce an apparent Lyo LF that matches that 
from the full RT model. The exp(— t,^) RT model with open triangles adopts a larger intrinsic Lyo lino width so that it roughly produces 
an apparent Lya LF matching that from the full RT model. 



D. EFFECTS OF THE INTRINSIC Lya LINE PROFILE AND THE DISPERSION BETWEEN INTRINSIC LUMINOSITY AND 

HALO MASS 

In this appendix, we first show the effect of the intrinsic Lya line profile on the scattered Lya emission and discuss 
how it may affect the clustering of LAEs. We then investigate how a scatter in the intrinsic Lya (or UV) luminosity 
and halo mass relation may change the clustering effects caused by the Lya RT. 

As mentioned in Paper I, one main uncertainty in the the RT model of LAEs is the initial or intrinsic Lya line 
profile (after Lya photons escape the ISM). In our current setup, the profile is assumed to be Gaussian with the 
width determined by the thermal temperature of the host halo. In reality, the shape and width of the intrinsic Lya 
line are determined by gas dynamics in galaxies and affected by galaxy disk rotation, galaxy merging, and galactic 
wind. As a result, the Lya line can be broadened, become asymmetric, and/or have a shift in wavelength. All of these 
effects may change the strength of the coupling between observed Lya properties and circum-galactic and inter-galactic 
environments, and therefore have an impact on the clustering effects discussed in this paper. 

A thorough investigation on the effect of the intrinsic line profile is out of the scope of this paper. Here we present a 
test to show how sensitive the observed Lya emission is to the change in the initial Lya line wavelength. We perform 
RT calculations for an individual source in a halo of 10^^ Mq and consider cases with different initial values of Lya 
wavelength. For each case, the initial Lya photons all have a single wavelength with the luminosity normalized to be 
the intrinsic Lya luminosity of the source. That is, the line follows the Dirac S function. 

In the left panel of Figure [211 the Lya surface brightness profile of the source is shown for a few cases. The initial 
Lya line wavelength is denoted by the wavelength shift relative to the rest-frame Lya line center, in units of the halo 
thermal velocity dispersion a ~ 70km s~^. 

On average, bluer initial Lya photons lead to less concentrated surface brightness profile, since these photons need 
to travel farther in the IGM before they redshift into the line center to encounter significant scatterings. Redder 
initial photons make the profile more concentrated and more point-like, A small fraction of the initial red photons 
can blucshift to the line center by encountering scatterings in the infall region around the halos, which makes the 
profile still appear to be extended on large scales. The solid curves in the left panel of Figure [24] show that the profile 
is insensitive to the initial Lya wavelength if the initial shifts of photons are within the ±3cr range. These photons 
encounter a lot of core scatterings and redistribute their wavelength in the ±3(7 range, and therefore they lost their 
memory of the initial states. 

In the right panel, we show the Lya flux in a central 3". 5 diameter aperture for different choice of the initial Lya 
wavelength. The two curves in the upper panel are fluxes observed in opposite directions. The difference in the two 
curve tells us how anisotropic the scattered Lya emission is and provides an estimate of the strength of the coupling 
between environments and scattered Lya emission through RT. If the initial shift of photons is smaller than 3.5a, the 
flux from the central aperture is strongly suppressed. Meanwhile, there is a large difference in the fluxes observed from 
the two opposite directions. Both indicate the strong coupling between the observed Lya emission and circum-galactic 
and inter-galactic environments. For photons with initial shift larger than ^ 3.5(t, a fraction of photons can be out 
of resonance in the circum-galactic and inter-galactic media, weakening the coupling. However, the fluxes in opposite 
directions still show '^20% difference. 

The above test seems to suggest that the environment-dependent RT effect becomes weak if the initial line shifts is 
larger than 3.5 a 250kms~^) for ha l os of 10^^h~^ Mp,. Galact ic winds can have the effect of shifting Lya emission 
toward red (e.g., Verhamme et al. '2006': ' Diikstra fc Wvithd 120101 ). Observationally, galactic winds are ubiquitous in 
star-forming galaxies. .Steidcl ct al.. (2010i) find that the interstellar absorption lines and the Ha line have a mean shift 
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Fig. 24. — Effect of initial Lya line wavelength shift on the Lya flux distribution. The left panel shows the surface brightness proflles of 
the scattered Lya emission for a source residing in a W^^h~^ Mq halo, chosen from the simulation box. For each profile, the initial Lya 
line from the central source has a single wavelength, denoted by the wavelength shift with respect to the rest-frame line center in units of 
the halo thermal velocity dispersion a ~ 70kms~^. The right panel shows the Lya flux in a central 3". 5 diameter aperture as a function 
of the initial Lya wavelength shift. The two curves in the upper panel arc for fluxes observed in opposite directions, and the curve in the 
lower panel shows their ratio. 
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Fig. 25. — Effect of scatter between intrinsic Lyo luminosity ^intrinsic and halo mass on the 3D redshift-space 2PCFs of LAEs. A 
Gaussian distribution is introduced for log ^intrinsic with standard deviation triog and fixed mean. The LAE sample is constructed by 
selecting LAEs above a threshold in observed Lya luminosity so that the number density is 2 X 10~'^/i'^Mpc~'^. From left to right, the cases 
for a-iog£=0, 0.5, and 1 are shown, respectively. See the text for details. 

of ~ 160km s"-'^ with a large scatter for z ~ 2-3 Lyman break galaxies with baryon mass of 10^" — 10^^ Mq. Such a wind 
velocity may be able to shift the Lya line by a few hundred kms~^. However, the outflowing neutral gas of the wind 
is observed to be usuall y collimated in a bipolar fashion (e.g. JBland fc TullvlllQSSi: iShopbell fc Bland-Hawthornlll998l: 
IVeilleux fc R"{I^[2002h . Lya emission escaping in directions other than the wind is expected to have smaller redward 
shift, which can still probe the circum-galactic and inter-galactic environments through scatterings. Therefore, it is 
likely that the clustering features we study in this paper persist in the presence of galactic wind. Radiative transfer 
modeling of individual galaxies in high resolution simulations with galactic wind prescription will help to further shed 
light on this issue. 

We now turn to discuss whether a dispersion in the intrinsic Lya (UV) luminosity and halo mass can substantially 
weaken the RT caused clustering effects. With the star formation recipe in the cosmological reionization simulation, 
whose output we use for the RT calculation, the SFR and halo mass appear to be tightly correlated. In reality, star 
formation may be more stochastic as a result of feedback, merging, gas accretion, and so on, and the SFR (or intrinsic 
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Fig. 26. — Effect of scatter between intrinsic Lya luminosity ^intrinsic and halo mass on the bias factors of LAEs. A Gaussian distribution 
is introduced for log ^intrinsic with standard deviation o-jog and fixed mean. The left and right panels are for absolute bias factor and 
that relative to halos, similar to Figure [TSl The bias curves are shown for both LAE and S-LAE samples, with (Tjog j^=0, 0.5, and 1 (from 
thick to thin curves), respectively. 

Lya luminosity) at fixed halo mass may vary a lot from halo to halo. At a fixed observed Lya luminosity, in addition 
to the RT caused mixing of host halo masses, we would have further mixing from the variation in the SFR. To see 
whether the stochastic star formation can wash out the clustering effects, we perform a simple test. 

We introduce a log-normal dispersion in the intrinsic Lya luminosity ^intrinsic and halo mass relation, characterized 
by the standard deviation aiogL of log ^intrinsic- To obtain the observed Lya luminosity, we assume that the observed- 
to-intrinsic Lya luminosity ratio does not change for each individual halo. Strictly speaking, this assumption is not 
accurate, since the radius of the contour of the detection threshold surface brightness (see Paper I) changes as one 
varies the intrinsic luminosity of a source. To be accurate, one needs to redo the expensive RT calculation. However, 
the assumption suffices for our purpose to investigate the effect of the scatter. We construct LAE and S-LAE samples 
with different thresholds in the observed Lya luminosity. 

Figure [25] shows the 3D redshift-space 2PCFs for n = 2 x 10~^/i'^Mpc~'^ LAE samples, with (TiogL=0 (no scatter, the 
default case in this paper), 0.5, and 1. At a fixed number density, a larger scatter brings a larger fraction of sources 
in low-mass halos into the sample, so the sample of LAEs would on average have a lower bias factor. We see that 
the clustering amplitude, indicated by the position of the thick solid curve, decreases as criogL increases. However, 
the elongation pattern along the line of sight, which is an indication of the environment dependent RT effect, remains 
clearly visible even for the case of CTiogi = 1 (i.e., one order of magnitude variation in ^intrinsic)- 

In Figure [26l we show the bias factor of LAEs and S-LAEs as a function of sample number density. From the left 
panel, the range of weak dependence of LAE clustering on sample number density tends to shift to lower number 
density for larger scatter CTiogL- At higher number density, the weak dependence breaks down, and the bias factor 
shows a similar slope as the halo samples. The right panel displays the bias factor relative to the halo sample. It turns 
out to be always true that LAEs are more strongly clustered than the corresponding S-LAEs, which means that the 
RT caused selection effect remains strong even for the case of large scatter. 

Our tests demonstrate that a large dispersion (up to ciogL = 1) in the intrinsic Lya luminosity and halo mass 
relation cannot completely wash out the RT caused clustering effects studied in this paper. 
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